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Preface 



Hybrid systems are interacting networks of digital and continuous systems. Hy- 
brid systems arise throughout business and industry in areas such as interactive 
distributed simulation, traffic control, plant process control, military command 
and control, aircraft and robot design, and path planning. Three of the funda- 
mental problems that hybrid systems theory should address are: How to model 
physical and information systems as hybrid systems; how to verify that their be- 
havior satisfies program or performance specifications; and how to extract from 
performance specifications for a network of physical systems and their simulation 
models digital control programs which will force the network to obey its perfor- 
mance specification. This rapidly developing area is at the interface of control, 
engineering and computer science. Methods under development are extensions of 
those from diverse areas such as program verification, concurrent and distributed 
processes, logic programming, logics of programs, discrete event simulation, cal- 
culus of variations, optimization, differential geometry, Lie algebras, automata 
theory, dynamical systems, etc. 

When the first LNCS volume Hybrid Systems was published in 1993, the 
effect was to focus the attention of researchers worldwide on developing theory 
and engineering tools applicable to hybrid systems in which continuous processes 
interact with digital programs in real time. At the time of publication of this 
fifth volume, there is general agreement that this is an important area in which 
mathematics, control engineering, and computer science can be fruitfully com- 
bined. There are now hybrid system sections in many engineering and computer 
science international meetings, hybrid systems research groups in many universi- 
ties and industrial laboratories, and also other excellent series of hybrid systems 
conferences. 

The impetus for this volume was the Fifth International Hybrid Systems 
Workshop held in Xotre Dame, Ind., USA, September 11-13, 1997. Previous Hy- 
brid Systems Workshops have taken place at MSI/Cornell (June 10-12, 1991), 
the Technical University Lyngby, Denmark (October 19-21, 1992), MSI/Cornell 
(October 28-30, 1994), DIMACS/SYCON Rutgers (October 22-25, 1995), and 
MSI/Cornell (October 12-14, 1996). The four volumes arising from these work- 
shops were: Hybrid Systems, Springer- Verlag LNCS 736 (1991 and 1992 work- 
shops); Hybrid Systems II, Springer- Verlag LNCS 999 (1994 workshop); Hybrid 
Systems III, Springer- Verlag LNCS 1066 (1995 workshop); and Hybrid Systems 
IV, Springer- Verlag LNCS 1273 (1996 workshop). All papers were fully refereed 
and selected from those submitted in a call subsequent to the Fifth International 
Hybrid Systems Workshop. 

Here are brief summaries of the papers which have been included. 

— Chen and Hanisch present a method for synthesizing hybrid feedback policies 
based on predicate invariance over the hybrid state space of the system. 

— Chutinan and Krogh develop the machinery for constructing approximat- 
ing automata for continuous systems where the continuous dynamics are 
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defined by convex polytopes in the space of derivatives of continuous space 
trajectories. 

— Davoren proposes the propositional modal /x®calculus - which subsumes 
most known temporal and modal propositional logics - as a broad logical 
framework for the formal analysis and verification of hybrid systems. Over 
transition system models equipped with a topological or metric structure, 
the logic can express continuity properties of transition relations and metric 
tolerance properties such as “being within distance e” of a set of states, thus 
allowing formal verification of robustness and stability properties of hybrid 
dynamical systems. 

— De Schutter and De Moor extend the Extended Linear Complementary Prob- 
lem algorithms previously used for discrete event systems to analyze some 
classes of hybrid systems. Their case study is of a traffic-light controlled 
intersection. 

— Ferreira and Krogh present the results of simulations based on a neural 
network model of controller scheduling for real-time switched systems. The 
switching strategy selects the current controller based on neural network 
estimates of the future system performance for each controller. 

— Gao and Xu model fault diagnosis and isolation as a hybrid system and 
express system specifications in the duration calculus, a dense time tempo- 
ral logic. An extension of program logic is used as a framework for formal 
verification. 

— Kohn, Nerode and Remmel introduce models for multiple sensor fusion of 
data and for synchronization of sensors by a Noether algorithm in the cal- 
culus of variations as well as a multiple agent hybrid sensor architecture for 
such problems. 

— Koutsoukas and Antsaklis use a class of timed Petri nets for supervisory 
control of hybrid systems. When the continuous dynamics arc described by 
first order integrators, this is a linear programming problem. 

— Kowalewski, Stursberg, Fritz, Graf, Hoffman, PreuBig, Remelhe, Simon and 
Treseler explain and compare eight computer tools - six simulation packages 
and two verification tools - with respect to validation of logic control pro- 
grams for continuous processes, with a two tanks problem as their benchmark 
example. 

— Lafferriere, Pappas and Sastry use recent results in the model theory of 
first-order structures over the real numbers to establish the existence of fi- 
nite bisimulation quotients for certain classes of planar hybrid systems, thus 
proving the decidability of verification problems for such systems. 

— Lemmon and Bett build on recent results on multiple agent linear control 
for systems satisfying a bounded amplitude performance constraint, to give 
a method of extracting a timed automaton as a logical model of specified 
switched system behavior. 

— Mosterman, Zhao and Biswas give semantics and simulation algorithms for 
a class of dynamical systems operating in so-called “sliding regimes.” With 
time and space scale abstractions, these become hybrid systems with chat- 
tering. 
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— Nadjm- Tehran! gives a model of time deterministic hybrid systems in which 
transitions can have different delays and gives a transition system semantics 
closed under various operations, including parallel composition. 

— Neller transforms initialized bounded-time safety problems for hybrid sys- 
tems into global optimization problems starting with an initial safety es- 
timate, and compares the results with simulated annealing and multi-level 
single linkage methods. 

— Parisini and Sacone analyze mode switching control policies for switched 
systems as two- level hybrid control schemes; they prove an invariance result 
for such schemes and report on simulation results. 

— Raisch, Klein, O’Young, Meder and Itigin give a methodology for approxi- 
mating continuous plant models by non-deterministic discrete automata and 
synthesizing discrete supervisory control for the approximation. The method- 
ology is illustrated by two examples from process control. 

— Ronkko and Ravn extend the action system framework for distributed and 
reactive systems to hybrid systems with differential equation actions and evo- 
lution guards; an extended notion of parallel composition for hybrid action 
systems is also developed. 

— Seibel, Farines and Cury describe a methodology for the design of flight plans 
for rotary-wing unmanned aerial vehicles based on formal verification using 
linear hybrid automata. 

— Skafadis, Evans, Mareels and Nerode present a solution to certain problems 
in mode switching controller design for stochastic dynamical systems with 
quadratic cost design, using dynamic programming to extract the desired 
control. 

— Wang, Khargonekar and Beydoun investigate robust control of hybrid sys- 
tems in the presence of modeling errors and structural uncertainties. For a 
class of non-linear hybrid systems, they develop a design methodology for hy- 
brid state feedback based on system performance and prove robust stability 
results. 

— Wong-Toi reduces problems of initialized rectangular automata to linear hy- 
brid automata, yielding semialgebraic algorithms for analyzing slope pa- 
rameters. Automated analysis is used to extract bounds on independent 
clock drifts in an audio control protocol. The emptiness problem for slope- 
parametric rectangular automata is proven undecidable. 

— Yu and Chen present a control framework for interval temporal systems, 
using duration calculus to express properties of open- and closed-loop be- 
havior. 

— Zhao, Loh and May describe their phase-space nonlinear control toolbox 
for synthesizing and evaluating control laws for a wide class of nonlinear 
systems. The maglev project is used as a testbed. 
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Predicate Invariance* 
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Abstract. A hybrid system is modeled by a hybrid automaton and its 
control specification is given by a predicate defined on the hybrid state 
space of the system. The problem is to find a hybrid state feedback policy 
that enforces the closed-loop system satisfying the specification. In this 
paper, we present a control synthesis method for the problem based on 
predicate invariance. 



1 Introduction 

Hybrid systems consist of a continuous time system and a discrete event system 
that interact and influence each other. Many artificial systems belong to the 
hybrid system family, instances of these systems include logic-based switching 
control systems, intelligent vehicle/highway systems and chemical batch pro- 
cesses. A growing need for modeling, analysis and design of hybrid systems in 
practice has attracted many researchers’ attention in control and computer sci- 
ence communities, and a considerable effort has been made to develop theoretical 
frameworks and models for such systems. 

Most work on hybrid systems in past was focused on modeling and analysis 
(verification), relatively little on control synthesis. Alur et. al. [ACH95] proposed 
a symbolic model-checking method for verification of linear hybrid systems based 
on reachability analysis over the infinite state space by iteratively computing sets 
of states. With this, a software tool HyTech was developed for analysis of the sys- 
tems [HHH95]. The method was then extended to slope-parametric rectangular 
automata [H97]. There also have been papers dealing with analysis of nonlin- 
ear hybrid systems, but most of them addressed approximation-based methods 
[CK97] [RY97]. For control synthesis, Tittus et. al. [TE94] proposed a method 
for integrator systems (a class of linear hybrid systems) based on iteratively 
computing extended jump sets, like the model-checking method of [ACH95]. A 
specific control synthesis problem of hybrid systems was formulated as an opti- 
mal control problem with both discrete and continuous variables in Branicky et. 

* This research was supported by Alexander von Humboldt Foundation 
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al. [BM95], but generally the optimal control problem is hard to be solved. Stiv- 
er et. al. [SAL95] proposed a hybrid control system synthesis approach based on 
natural invariants, however it requires quantizing and bounding the state space 
so that the approach can be implemented by a computer algorithm. In addition, 
some work on hybrid systems was focused on stability issue (see Branicky et. al. 
[B94] and Pettersson et. al. [P96]). 

For general hybrid systems, it seems computationally intractable (undecid- 
able) for analysis and control synthesis. However, it may be possible to provide 
some theoretical results with which and with specific structures of real prob- 
lems the problems can be successfully solved in finite computation time. In this 
paper, we present a method for control synthesis of hybrid systems based on 
predicate invariance. The hybrid system we consider is assumed to be described 
by a hybrid automaton similar to that proposed by Tittus et. al. [TE94], and 
the specification for the system (or the control objective of the system) is given 
by a predicate defined on the state set of the hybrid system (each state in the set 
consists of a discrete state and a continuous state), which is usually characterized 
by a collection of linear inequalities. 

The aim is to find a hybrid control policy that makes the closed loop system 
satisfying the specification. We first introduce control-invariant predicates for 
the hybrid system and then prove that, for a given predicate, the necessary and 
sufficient condition for the existence of a control policy that verifies the predicate 
is that it is control- invariant. With this result, we propose a fixpoint algorithm 
for computing the extremal control-invariant predicate. On the basis of this, 
a desired control policy for the hybrid system may be formally synthesized. A 
control synthesis problem of an evaporation process in a benchmark batch plant 
is given to illustrate the algorithm. 

2 Hybrid Plant Model 

An open loop hybrid plant is described by a seven-tuple 



( 1 ) 

where the components can be divided into three groups: 

a) Discrete event part 
Q is a finite set of discrete states q 

Sh is the set of hybrid plant events a. This set Sfi — Sp U Sc is partitioned 
into the set of physical events Sp and the set of control events Sc. The events 
in Sp occur as a result of the continuous state evolution and are uncontrollable, 
while the control events in Sc are external inputs and are controllable. 

^ : Q ^Sh ^ Q ( 2 ) 

is the (discrete) state transition function for the hybrid system . 
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b) Continuous part 

X is a subset of the n-dimensional real valued state space, and x{t) g X 

is the corresponding state vector. 

The function / : X ^ X is the vector field of the state vector x, that is 

x{t) = (3) 

where q E Q is the present discrete state of the hybrid system. 

The function g : X -^Q -^Sh X specifies discontinuous jumps in the state 
vector at discrete state transitions, i.e., 

x{t^) ^ g{x{t~),q,a) (4) 

where t is the time when the event occurs at state q. 

c) Physical event generator 



2^- (5) 

The function 7 p implicitly defines a set of regions in the continuous state 
space for each discrete state, i.e., = {x g X\(t g 7 p(g,x)},fj g Xp. When the 

continuous state x{t) enters one of the regions at discrete state g, i.e., x{t) g i?® 
for some cr, the corresponding physical event <j will occur immediately if no 
conflict with other events is existed. It should be noted that there may exist 
more than one events a such that x(t) g i?® . In this case, only one event of them 
can occur immediately, i.e., they are conflict events at the discrete state. 

3 State Feedback Controller 

The state feedback controller for the hybrid plant we consider is defined as 

7e : Q ^ 2^<= (6) 

It can be regarded as an event generator for control events. The closed loop 
system CHP — with the state feedback control structure is shown in 

Fig. 1. The behaviour of the closed loop system is as follows: When the system is 
at a discrete state g, its continuous state evolves according to differential equation 
(3). A physical event (resp. a control event) occurs as soon as the continuous 
state enters one of the regions specified by physical event generator jp (resp. 
control event generator 7 c ). A discontinuous jump in the continuous state may 
occur with the occurrence of the event, which is governed by the function (4). 

Given an open-loop plant model and a specification for the desired closed- 
loop behavior, the aim is to obtain such a control event generator which satisfies 
the specification for the closed loop system. In this paper, we assume that the 
specification is given by a predicate (a state subset) P : Q -^X {0, 1} defined 

on the hybrid state space Q -^X . The predicate is usually defined through a set 
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State Feedback 
Controller 



Hybrid Plant 




Fig. 1. Closed loop system 



of mode invariants and/or system invariants [TE94], where the mode invariants 
are specified by labeled inequalities 

Ptnviq) ■■ gq{x) ^0 (7) 

In the following, the expressions P{q, x) — 1 (resp. P{q, x) — 0) and (g, x) C 
P (resp. (g,x) / P) are thought as equivalent ones, and we say (g,x) satisfies 
(resp. does not satisfy) predicate P if P{q,x) = 1 (resp. P{q,x) = 0). 

4 Control-invariant Predicate 

The following notations will be used in the sequel 

Pq — x) £ P}: the continuous state set at discrete state g, given by the 

predicate P. 

P^ and dPq’. the internal point set and the boundary point set of Pq respec- 
tively. 

Sq,x’- the set of events that are plant-enabled at state (g, x), i.e., Sq,x = {cr £ 
Sp\5{q, cr)! A cr £ 7 p(g, x)} U {cr £ Sc\d{q, cr)!}, where 6{q, cr)! means that 5{q, a) 
is defined. 

x{t,x,q)-. the continuous state trajectory of differential equation (3) with 
initial state x{0,x, q) = x. We use symbol x to represent a state trajectory and 
X to represent a point at the state trajectory. 

FD{x, Pq): the set of feasible directions at x for set Pq. Here, a vector d is 
said to be a feasible direction at x for set Pq, if and only if 3e > 0,x + rd £ 
Pq, VO < r < £. 

= {x|x £ Pq\/{x £ dPqhf{x, q) £ FD{x, i/j)}} is the set of all continuous 
states from which the continuous state of the system will remain in set Pq for a 
sufficiently short period at least. 
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With these notations, we define the predicate: 



Da{q,x) 



1 S{q,a)\ Aa e jp{q,x) 

0 otherwise 



( 8 ) 



Dcr is the predicate identifying the set of states at which event a is plant- 
enabled. 

We define the predicate transformations: 



wpc{P){q,x) 



1 S{q,a)\ Act G Ap{q,x) A g' = 5{q,a) Ag{x,q,a) G P/, 

0 otherwise ’ 

a € Up (9) 



wpa{P){q,x) 



1 5{q,a)\ A q' ^ d{q, a) A g{x, q, a) G P;j, 

0 otherwise 



cr G (10) 



wp„{P) is called the weakest precondition of P under cr. wpa{P){q,x) = 1 if 
and only if 1) a is plant-enabled at state (g, x) and the occurrence of event a 
leads to a new state which satisfies predicate P. 2) From the new state, the state 
of the system will keep satisfying P for a sufficiently short period at least. Note 
that the enabling condition for physical event a is d(q,a)! A cr G 'Yp{q,x) but 
S{q,a)\ for control event cr. This is why the formulation of wprj(P) for physical 
events and that for control events are little different. 



wlp„{P) = wpa{P)y (11) 

wlpa-{P) is called the weakest liberal precondition of P under cr. wlpa-{P){q, x) 
= 1 if and only if cr is not plant-enabled at state (g, x) or from the new state 
resulted from the occurrence of a at state {q,x), the state of the system will 
keep satisfying P for a sufficiently short period at least. 



wpciock{P){q,x) 



1 there is T ^0 and a G Piq,i{T,x.q) 

such that P{q,x{t,x,q)) = 1 Vt G [0,T] 
and wp^{P){q,x{T,x,q)) = 1 
0 otherwise 



(12) 



wPclock{P) is called the weakest precondition of P under clock {time advance). 
wpciock{P){q, x) = 1 means that at state (g,x) the system can be switched to a 
new mode (a new discrete state) so that the resulted new state satisfies P for a 
sufficiently short period at least, or starting from state (g,x) the system can run 
for a period of time, during the period the state of the system keep satisfying 
P. and at the end of the period the system can be switched to a new mode with 
the resulted new state satisfying P for a sufficiently short period at least. More 
formally, wpciock{P){q,x) can be rewritten as 



wPciock{P){q, a;) = „ ^^wp„{P){q, x{T, x, q))AAj^QP{q, x{t, x, q))} 

(13) 
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To exclude the physically unrealistic possibility that the time advance might 
be preempted indefinitely by repeated occurrence of a set of events without time 
elapse, we impose a technical condition that any two events in the hybrid system 
can not occur successively without time elapse {Assumption A). 

Definition 1. A predicate P is said to be control-invariant if 

P ^{f^c7€SpWlpa{P)} A WPclock{P) (14) 

^From the definition, if a predicate P is control-invariant, then for any {q, x) £ 
P, we have: 1) if there is a physical event a that is enabled and fired at state 
(q,x), the resulted new state will satisfy P as well. 2) at state {q,x) the system 
can be switched to a new mode (a new discrete state) so that the resulted new 
state satisfies P, or starting from state {q,x) the system can run for a period of 
time, during the period the state of the system keep satisfying P, and at the end 
of the period the system can be switched to a new with the resulted new state 
satisfying P for a sufficiently short period at least. 

The main result of this paper is as follows 

Theorem 1. Given a predicate P, there is a state feedback controller of the form 
( 6) such that for the closed loop system, P remains invariantly true whenever it 
is initially satisfied, if and only if P is control-invariant. 

Proof : 

IF. Let Pc{q,x) = {ct £ Sc\5{q,a)\ Aq' = 6{q,a) Ag{x,q,a) £ P^,}, 

Pp{q,P) = {o- C Sp\5{q,a)\ Act £ Jp{q,x) A q' ^ d{q,a) Ag{x,q,a) £ P;^,}, 

we first prove Pc{q,x) U Pp{q,x) 7 ^ 0 for any x £ Pq\Pj {Property A). 

/,From control-invariance of P, we have {q, x) £ wpdo cfe(P). Since x £ Pq\Pg,, 
from the definition of wpdock{P), there is ct £ Sh which is plant-enabled so 
that the resulted new state {q' , g{x,q,a)) by firing ct at state {q,x) satisfies 
g{x,q,cj) £ P^,, i.e., ct £ Pc{q,x) UPp{q,x), thus Pc{q,x) UTp(q',x) ^ 0 . 

Dehne a state feedback controller as follows: 

7c : Q -^X 2^'= 

7 c (g, x) -^Pc{q, x),if X E P/ 

7c{g, x) ~^Pc{q, x) and 7 c(g, x) U Pp{q, x) 7 ^ 0, if x £ P,\P/ (15) 

Such a state feedback is existed because of Property A. For the closed loop 
system CHP = {HP,jc) and a state {q.x) £ P, there are two cases. One is that 
there is no event ct £ Eh enabled at the state. For this case, x £ pf . The system 
can run for at least a period of time, during the period the state of the system 
keep satisfying P. The other is that there is an event ct £ Eh enabled at state 
{q,x). For this case, ct £ Pc{q,x) or ct £ Pp{q,x), the firing of each of them at 
state (g, x) will lead to a new state {q' , g{x,q, a)) with g{x,q,a) £ P^,, so the 
new state also satisfies P. Summarily, P remains invariantly true for the closed 
loop system whenever it is initially satisfied. 
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OXLY IF. If P ^ {f\a€SpW^P(r{P)} A wpciock{P) IS not true, then we have 
(g,x) e P, such that {q,x) ^ A„(:E^wlpa{P) or {q,x) ^ wpdock{P)- H {q,x) ^ 
Aa^EpWlpa-{P), then there is cr G Yp that is plant-enabled at state {q,x) such 
that g{x, q, a) ^ P^,, q' = 6{q, a). Since a is uncontrollable, recalling the assump- 
tion A, we assert that the truth of P will be violated during the state evolution of 
the closed loop system for any state feedback controller. For {q,x) ^ wpdock{P), 
we can similarly prove the assertion. However, the assertion is contrary to the 
premise. 

^From the proof of Theorem 1, if a predicate P is control-invariant, then a 
state feedback controller (a control event generator) which satisfies the specifi- 
cation can be synthesized according to (15). 

To check if a predicate P is control-invariant, we are required to compute 
wlpa{P) for each a E Up and wpdock{P)- The computation of wlpa{P) is easy 
but of wpdock{P) not. However, with some conditions on vector field f{x,q) of 
the continuous state at each discrete state q, on physical event generator ■jp and 
on predicate P, it is still possible to compute wpdock{P) efficiently. 

For example, suppose that the solution x(t, x, q) of the differential equation 
(3) can be explicitly obtained and that f{x,q), jp and P satisfy the following 
monotonicity conditions: 

1) For each (q,x), a G Sq^£{t,x,q) some t ^0 implies a G Sq.x{t',x.g) 
any t' —tt, 

2) For each {q,x), P{q,x) = 1 and P{q,x{t,x,q)) = 1 for some t ^0 imply 
P{q,x{t' ,x,q)) — 1 for any t' 

Then, wpdock{P) can be computed in the following way: 

1) For each q, <J £ Sq — {<7 & Sh\^{q, cr)\}, we compute tq^q{x) — min{t\a G 
P'q,x(t.x,q) ^ ^ 

2) For each q, we compute tq p{x) — rnax{t\P{q,x{t' ,x,q)) — l,for any 0 ^ 
t' 

3) wpdock{P){q,x) = 1 if tq^^{x) ~^tq^p{x) and there is tq^„{x) ^tq^p{x) 
such that wpdock{P){q, X, q)) = 1, otherwise wpdock{P){q, x) = 0. 

The applicability of the above procedure depends on whether we can obtain 
explicit expressions of tq^„{x) and tq^p{x) . 

5 Extremal Control-invariant Subpredicate 

Theorem 1 in last section does not imply that if predicate P is not control- 
invariant, then we can not find a state feedback controller such that for the closed 
loop system, P remains invariantly true for some initial state. But in this case, 
the initial state must be taken from a control-invariant subset (subpredicatc) of 
P. There are probably many such control-invariant subsets. Among them, we 
pay attention to the ones which are good control-invariant approximations to P. 

Let Q be the set of all predicates defined on Q -^X. For P E Q, we define: 

CI^{P) = {P'\P' G Q,P' ~^P and P' is control O invariant} 
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CI^{P) is the set of control- invariant predicates on Q that are stronger 
than P. It can be easily proved that CI^{P) is nonempty and is closed under 
arbitrary disjunctions (unions). With this property, we can assert the existence 
and the uniqueness of the maximal elements of C'/<(P). Let P^ denote the 
maximal element. Pl can be thought as the best control-invariant approximation 
to P among the predicates which are stronger than P. 

We can develop a fixpoint algorithm for the computation of P^. For this 
purpose, we introduce the map: 



H -Q^ Q 

H{P') = P' A {Kes,wlPa{P')} A wp,iock{P') (16) 

and define the sequence of predicates: 

Po = P,Pfc+i =P(Pfc),fc = 0,l,2,... (17) 

It can be proved that the sequence defined in the above is monotone decreas- 
ing and pi = limk^ooPk- With this result, we can compute pl by the iterative 
process given by (16) and (17). It should be noted that it is not clear at present 
if the fixpoint algorithm always terminates in finite steps. 

The notion of predicate invariance and the algorithm presented in this paper 
can be regarded as an extension of these of [RW87] for discrete event systems 
to hybrid systems, so readers please refer to [RW87] for the proofs of the results 
in this section. 

For a linear hybrid system and a predicate given by a linear formula, it can be 
proved that the predicates Pfe, k — 0,1,2, . . . generated during the run of the fix- 
point algorithm are definable as unions of convex polyhedra in multidimensional 
real space, so that a symbolic computation algorithm like the model-checking 
procedure implemented in HyTech [HHH95] can be designed to automatically 
find pi using computer. It is very difficult to implement such a symbolic com- 
putation algorithm for general hybrid systems, however, in the next section, we 
will show by an example that even for nonlinear hybrid systems, it is possible 
to efficiently compute pl by using the theoretical results proposed in previous 
sections and by taking account of specific structure of the system in the example. 

6 Example 

6.1 Process description 

We use a part of a benchmark plant described in [HK94] as an example. The 
flowsheet is depicted in Fig. 2. B is an evaporator that transforms low concentra- 
tion salty water to high concentration salty water by means of evaporation, VI 
and V2 are the valves for feed and discharge of salty water respectively, pipe PI 
is an outlet for steam generated by B (PI is connected to a condenser), H is the 
(electrical) heating. There are continuous sensors LIS, TI and QIS measuring the 
level, the temperature and the concentration of salty water in the evaporator. 




Control Synthesis of Hybrid Systems Based on Predicate Invariance 



9 




Fig. 2. An evaporation process for dissolved salty water 



The recipe of the evaporation process is as follows: An amount of low concen- 
tration salty water is first filled into B, and is then heated to evaporate. During 
the evaporation process, the steam generated by B goes out from B through pipe 
PI. The process will be terminated when the concentration of the salty water 
reaches to a given value. The salty water with given concentration is afterwards 
discharged through valve V2 as product of the process. 

For simplicity of modeling, we assume 

a) There is an underlying controller that prevents valves VI and V2 from 
being opened concurrently, and from being opened while the salty water in B is 
being heated. 

b) The time lag of the heating is negligible. 

c) During the filling, discharging and heating, the heat exchange between the 
salty water and its environment is negligible. 

d) The concentration of the salty water in the whole evaporation process is 
fairly low so that the mass density, the specific heating capacity and the specific 
evaporation enthalpy of the salty water can be approximated by these of pure 
water respectively. 

e) The flow rate of valve VI is constant and the flow rate of valve V2 depends 

on the level of the salty water (cx where g is the gravity constant and x is 

the level). 

f) B is a standard cylinder 
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6.2 Hybrid model 

We need the following parameters to set up our hybrid automaton model of the 
process (see Fig. 3 ): 



ri: flow rate of valve VI ( 1 /s), suppose that ri is a constant 
?’2; flow rate of valve V 2 ( 1 /s), T2=k2^/gx , where is a constant, x is the 
level of the salty water, g is the gravity constant 
c: heat capacity of water (kJ/kgC®) 
e: evaporation enthalpy of water (kJ/kgC°) 

cq: concentration of the salty water filling into B through valve VI (kg/m^) 
Cd’- desired concentration of the salty water (kg/m^), Cd —^cq 
p : mass density of water (kg/m^) 

P\ power of the heating (kJ/s) 

S', square of the base of B (m^) 

p 

— 7770 , Cy.2 — 



P 

epS 



h: the maximum admissible level of the salty water in B (m) 

1 : the minimum admissible level of the salty water in B (m) 

Tq: temperature of the salty water hlling into B through valve VI (environ- 
ment temperature, C°) 

fc: a parameter for natural fall of the temperature (1/s) 



Events (transitions) for the process'. 

V|/ : open valve VI (control event) 

V/: close valve VI (control event) 

F: B is full and the valve VI is closed automatically (physical event) 

H+: turn on the heating (control event) 

Eva: the salty water begins to evaporate (physical event) 

D: the desired concentration is reached and the heating is turned off auto- 
matically (physical event) 

LB: the minimum level is reached and the heating is turned off automatically 
(physical event) 

V^ : open valve V 2 (control event) 

E: B is empty and the valve V 2 is closed automatically (physical event) 



Continuous state variables for the process: 
xi: level of the salty water in B (m) 

X2'. temperature of the salty water in B (C®) 

Xg: concentration of the salty water in B (kg/m^) 



Discrete states and their corresponding continuous dynamics for the process 

qo (initial state) : ii = 0, X2 = 0, ±3 = 0 (xi(0) = 0, X2(0) = Tq, ^3(0) = cq) 

qi (filling): = n, X2 = 0, X3 = 0 

q2 (ready to be heated): xi = 0, ±2 = 0, ±3 = 0 

qg (heating): xi = 0, ±2 = f/, ±3 = 0 

qi (evaporating): xg = ®a2, i'2 = 0 , xg = 
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(ready to be discharged): x\ — Q, X2 — k{To 0 X2), X3 = 0 

qe (discharging): xi = 0k2^gxi, ±2 = 0, X3 = 0 

Physical event generator: 

q = qi, xi = h: the salty water in B reaches its maximum level, event F 
occurs. 

q = qs, X2 = 100: the temperature of the salty water in B reaches lOOC®, 
event Eva occurs. 

q = qi^ xs = Cd’- the salty water in B reaches its desired concentration, event 
D occurs. 

q = q4, xi = 1 : the salty water in B reaches its minimum level, event LB 
occurs. 

q — qe, xi = 0: the salty water in B is completely discharged, event E occurs. 

Discontinuous state jumps: 

at transition E of state qe: 

X2 ^ To, xs Co 

It means that after the salty water in B is completely discharged, the plant 
returns to its initial state, a new cycle of the production then begins. 



E: x,=0 




^0 






02 



3 

qfj ^2- 






^ Eva 

LB:x,<=l ‘Is 




Fig. 3. Hybrid automaton model of the evaporation process 



6.3 Specifications for the process 

a) system invariants 

I -^xi —fh 
0 ^^3 ^^max 

where the first invariant specifies that the level of the salty water in B must 
be between its minimum value and maximum value, the second invariant speci- 
fies that a maximum concentration (cmax) of salty water must not be exceeded, 
otherwise the salty water in B may boil over (foam may arise and flow to the 
condensor) during the evaporation phase and may crystallize during the heating 
phase. We assume Cmax Cd, otherwise it is impossible to produce a batch of 
desired concentration without violating the system invariant. 
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b) mode invariants 

Cd® £ ~^X3 ->Cd + e at g = gs 
Cd®£ ->X3 -^Cd + e at g = ge 

where £ is the maximum admissible deviation between the concentration of the 
produced salty water and the desired concentration. The mode invariants specify 
that there should be no waste batch produced in the plant. 

6.4 Control synthesis based on predicate invariance 

According to the hybrid automaton model of the plant, it is clear that the 
system invariants of the plant have been obeyed, so we only need to consider 
the mode invariants of the plant. We try to find a maximally permissive state 
feedback controller that makes the closed loop hybrid system obey all of the 
mode invaraints. 

The mode invariants can be described by a predicate (a set equivalent to the 
predicate): 

P = {(g,x)|g G Q,x e X and Cd® £ -^xs -^Cd + e if g G {g 5 ,ge}} 

where Q = {go, 92, ^3, 94, ?5, ge}, A = {(xi, X2, X3)|0 xi h,To X2 
100, 0 ^X 3 ^1}. For simplicity, P will be represented by P = jgs, ge, Cd® £ ~^ 
X3 ^ Cd + £} V (go, 9i, 92, 93, 94}- Such a convention will be applied to other 
predicates to be appeared in the following as well. Note that with the convention, 
set (predicate) {90, 9i, 92, 93, 94, 95, 9e} = {(9, ^;)|9 G Q,x G X}. 

Note that 



Ap = (F, Eva, LB, D, E}, Sc 



{V+,Vr,H+,V2+} 



We have: 



wIpy{P) = wp-p{P)V 

= {90,91,92,93,94,95,96} 

wlpEva{P) = { 90 , 91 , 92 , 93, 94 , 95, 9e} 

wIpeb(P) = { 94,^1 ~^l,Cd®£ ~^X 3 -^Cd + e}V{g4,xi > 0^ { 90 , 9i, 92, 93, 95, 96} 
wIpb{P) = { 94 , Cd ® £ ^X 3 -^Cd + £} V { 94 , X 3 < Cd} V {90,91,92,93,95,96} 
wIpe{P) = { 90 , 91, 92, 93, 94, 95, 9e} 
wPy+{P) = { 90 } 

WPy-iP) = { 91 } 

WPR+{P) = { 92 } 

WPy+ (P) = { 95 , Cd ® £ ~^X3 -^Cd + £} 
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and then 



AffeSp wlpa{P) = {qi, Cals' e + e} V {94, xi > I, xs -^ca + e} 

V{(?o,<7i,<?2,g3,<75,<?6} 

Note that the solution of the differential equation corresponding to discrete 
state g4 (the continuous state trajectory at q^) is 

xi{t) = Sa2t + xi(0), 

X2{t) = X2(0), 

X 3 {t) = exp{ [ ^ — ^dT)x3(0), 

Jo 0Q!2T + Xi(O) 

where (a:i (0), 0:2(0), 3:3(0)) is the initial state at 54. xo{t) is monotonously in- 
creasing when Scx2t + a:i(0) ^0. 

At hybrid state (94, x) (x = (xi,X2,X3)), wpciock{P){q4, x) = 1 if and only if 
there is t ^0 such that for any t' G [0, t], (8ia2t' + xi -^l, Sa2t + xi — I, caSs ^ 

t 

expijg r^j^^Sp^dr)x3 -^Cd + £, or 0a2t' + xi exp(fg tfr)x3 -^Ca, 

exp(fg d.T)x3 = Cd- Since exp(fg dr) is monotonously increasing 

when Sa2t + xi ^0, we have: wpciock{P){q4,x) = 1 if and only if xi X3 ^ 
Cd + £ and there is t ^0 such that ®a2t+xi caSe -^exp(fg dr)xs 

Cd + 6 . Obviously, the necessary and sufficient condition for wpdock^P){q4, x) = 1 

is equivalent to xi x$ — >Cd + e, caS £ — »exp(Jg -a^r+xi because 

of monotonicity of exp( /g dr) . 

So, we have: 



WPclockXP) = {gs, 96, Crf ® £ -^X3 -^Cd + e} 

V{g4,xi -^Z,X3 ~^Cd + £,CdSe ^exp{ / 

Jo 

v{go,gi,g2,93} 

= {?5, 96, Cd ® £ ->X3 -^Cd + s} 

r 1 Xi , 

V{g4,Xi —^l,X3 ^Cd + £,CdS £ 
V{90,91,92,93} 



x-\_ —I 
«2 



CX2 



SCt2T + Xl 



dr)x3} 



and then 



Pi = H{Po) = H{P) = P A {^c€SJ,wlpa{P)} f\WPclock{P) 
= { 95 , 96, CdS£ ~^X 3 ^Cd + £} 

V{g4, Xl X3 ^Cd + £, (cd O £)/ ^XiX3} 

V{90,91,92,93> 

Similarly, we have: 



wIpf{Pi) = {90,91,92,93,94,95,96} 
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wlpEva{Pi)^{q 3 ,X 2 ^l 00 ,xi^l,xs^Cd + e,{cd<S'e)l -^xiXs}y{qs,X 2 < 100} 
v{go,q'i,<72,q'4,g5,g6} 

wlphB{Pl) = {g4, Xi XI -^l,X 3 ^Cd + £, {Cd ® s)l ^XiX^) V {g4, X\ > 1} 
V{go! 9ii 92, (/3i 95, 9e} 

= {g4, Xi =l,Cd^e ~^X 3 ^Cd + e} V { 54 , xi > 1} 

V{9o, 91, 92 , 93,95,96} 

wlpuiPi) = {94,a::3 ^Cd,xi ^Z,X3 + e, (q 0 ^xixs} V {q 4 ,X 3 < q} 

V{9o, 9i, 92, 93, 95, 9e} 

= { 94 , xi Cd ~^X 3 —>Cd + e} V { 54 , X 3 < Cd} 

V{9o,9i,92,93,95,9e} 

wIpe{Pi) = {90,91,92,93,94,95,96} 

wpciock{Pi) = { 95 , 96, Cd 0 e ^X 3 ^Cd + e} 

V{g4, 93, 2^1 X3 ^Cd + e, {cd 0 e)l ^xiX3} 
V{9o,9i,92} 

Thus, 

P2 = H'(Pi) = Pi A {/\(res„wlpa{Pi)} A WPclock{Pl) 

= { 95 , 96, Cd 0 £ ^ai3 ^Cd + e} 

V{94, 93, a^i X3 -^Cd + £, (cd 0 e)l -^XiXs} 
V(9o,9i,92} 

Similarly, we have: 

P 3 = { 95 , 96, Cd 0 £ ^X3 ^Cd + e} 

V{94, 93, 92, ^1, X3 -^Cd + e, (cd 0 e)l ^xixs} 

V(9o,9i} 

•P 4 = { 95 , 96, Cd 0 £ -^X3 -^Cd + £} 

V|94,93,92,9i,a:i ^/,X3 Cd + s , {cd e)l ^xixs} 

V{9o} 

P5^ Pi 

SO P5 is the maximal control- invariant subpredicate of P. 

With the maximal control-invaraint subpredicate, a (maximally permissive) 
state feedback controller for the hybrid system can be easily synthesized accord- 
ing to (15). 
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7 Conclusion 

In this paper, we provide a theoretical framework for control synthesis of hybrid 
systems based on predicate invariance. By a benchmark example, we show that 
it is possible to efficiently synthesize control laws for hybrid systems by using 
the theoretical results proposed in the paper and by taking account of specific 
structures of real systems. Further work is required to propose efficient control 
synthesis methods for some specific subclasses of hybrid systems in the light 
of the theoretical framework and to propose some approximation methods for 
symbolic computation of the extremal control-invariant subpredicate for general 
hybrid systems. 
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Abstract. Approximating automata are finite-state representations of 
the sequential input-output behaviors of hybrid systems characterized 
by threshold events that trigger discrete changes in the continuous dy- 
namic equations. Procedures proposed for constructing approximating 
automata require forward and backward mappings of sets of continuous 
state trajectories - mappings which are not available for arbitrary con- 
tinuous dynamics. This paper develops the foundations for constructing 
approximating automata automatically for hybrid systems in which the 
continuous dynamics are defined by convex polytopes in the vector space 
of the derivatives of the continuous state trajectories. The computation- 
s are illustrated for a simple example which also demonstrates the use 
of approximating automata to solve verification problems that may be 
intractable using fixed-point computations for linear hybrid automata. 



1 Introduction 

This paper concerns the generation of purely discrete models (finite automata) 
for a class of hybrid systems in which the continuous-state trajectories generate 
events when specified threshold hypersurfaces are reached. The continuous dy- 
namics are selected by a discrete input signal which switches only when threshold 
events occur. An algorithm was proposed and illustrated in [3] for computing 
approximating automata for such hybrid systems with arbitrary continuous dy- 
namics. The languages of the approximating automata contain all possible se- 
quences of pairs of inputs and threshold events that are valid for the hybrid 
system. The approximation can be refined iteratively, converging when possible 
to a finite automaton that generates the exact sequential behaviors of the hybrid 
system [9] . In [2] we proposed an alternative algorithm for constructing approx- 
imating automata for hybrid systems which can be refined more efficiently than 
the approximating automata in [3] . 

The purpose of this paper is to identify a set of hybrid systems for which 
the approximating automata can be constructed automatically. The algorithms 
for computing approximating automata rely on mappings that characterize the 

* This research was supported in part by DARPA contract F33615-97-C-1012. 
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forward and backward evolution of families of continuous-state trajectories. It is 
also necessary to compute the set of continuous states from which no events will 
be generated, called the set of null event states. For arbitrary continuous dy- 
namics, there arc no computationally effective ways to compute the forward and 
backward mappings and the set of null event states. In this paper we consider 
hybrid systems with continuous dynamics in which the derivative of the contin- 
uous state vector lies in a convex polytope, similar to the continuous dynamics 
in so-called linear hybrid automata [1]. For this case, it is possible to carry out 
all of the computations necessary to generate the approximating automata using 
standard computations on linear inequalities and convex polytopes. This paper 
demonstrates how this can be done in the context of the algorithm in [2]. The 
same computations could be used in the algorithm in [3]. 

Although many (but not all) of the computations discussed in this paper are 
similar to computations used in programs such as HyTech [1] for the verification 
of linear hybrid automata, approximating automata provide a different approach 
for hybrid systems analysis and synthesis. To perform verification when the 
logic for selecting the discrete input based on the threshold events is given, 
an approximating automaton can be generated hrst. Properties of the hybrid 
system can then be verified using tools for finite state systems, such as SMV [7] 
or COSPAN [6]. One approximating automaton may be used to verify several 
properties of the hybrid system using these faster, more efficient tools. As shown 
by the example in section 6, approximating automata may make it possible to 
verify properties for which the fixed-point computations for the linear hybrid 
automata will not terminate. 

Approximating automata can also be used for the synthesis of discrete con- 
trollers. For this application, illustrated in [3], the approximating automata rep- 
resent the sequential behaviors of the hybrid system under all possible choices 
of the discrete input at each threshold event. This model can be used as the 
plant to synthesize control logic based on the standard discrete event system- 
s theory [11]. Alternatively, an approximating automaton could replace hybrid 
system dynamics in parts of a large simulation model to enable much faster eval- 
uation of alternative designs. For both controller synthesis and simulation, the 
approximating automata provide a finite-state models of all possible sequential 
input-output behaviors of the hybrid dynamics which can be used as subsystem- 
s in larger systems of interconnected input-output models, facilitating modular 
analysis and design based on discrete techniques. 

The concept of approximating automata has also been developed by Raisch 
and O’Young for discrete-time hybrid systems [10]. Their algorithm for con- 
structing approximating automata is similar to the algorithm in [3], but the 
forward and backward mappings are given immediately by the discrete-time 
continuous state transition equations. For continuous-time hybrid systems, the 
approach of [10] would apply to sampled-data implementations where the sam- 
pling times are fixed and known a priori. In the continuous-time model considered 
in this paper, the ’’sampling times” are generated by the threshold events, which 
means the forward and backward mappings must be computed by integrating 
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the state equations. It is precisely this problem, computing the continuous s- 
tate trajectories, that is addressed in this paper for a particular class of hybrid 
systems. 

The remainder of the paper is organized as follows. The following section 
describes the class of hybrid systems to be considered. Section 3 defines the map- 
pings of continuous-state trajectories to be used in the construction algorithm, 
which is presented in section 4. Section 5 presents the details for calculating the 
mappings and null event sets for the class of linear hybrid systems defined in sec- 
tion 2. A simple example is presented in section 6 to illustrate the computation 
of approximating automata for verification. The concluding section summarizes 
the results in this paper and discusses current research directions. 



2 A Class of Linear Hybrid Systems 

In this paper we consider the class of hybrid systems illustrated by the block dia- 
gram in Fig. 1. The signal u(^is a right-continuous piecewise constant condition 
signal taking on discrete values in the finite set U . This signal is generated as the 
state of a condition/event (C/E) system whose state transitions are forced by 
the event signal v{^ a pointwise-nonzero n„-dimensional signal taking on values 
in {0, 1}"”. The initial state of the C/E system is given as u(O^) G U . The set 
of events that drive transitions in the C /E system are the nonzero values of the 
event signal w(dt that is, values in the threshold event set V = {0, 1}"” ® {0}"". 



convex 

polyhedron 




Fig. 1. Block diagram illustrating the class of hybrid systems being considered 



The block in Fig. 1 receiving the input signal u(^is a signal generator whose 
output signal x(^is a piecewise continuous signal with values confined at each 
instant to be in the set ^ i?"* , where u is the current value of the input 
signal u(^and is the dimension of the continuous state space. 
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We assume that for each u G U, is a convex polytope defined by the 
matrix- vector inequality 



F„ = {s G (1) 

where G ^ ^ i?'"" and integer m„ are given for each u. We refer 

to a set of the form (1) as a derivative constraint set. 

The integrator block in Fig. 1 operates on each component of its input signal 
to generate the continuous state trajectory x{^ Xq is the set of possible initial 
continuous states. The continuous state vector is multiplied in the gain block by 
the output matrix C G creating the continuous output trajectory y{^ 

The threshold events which constitute the non-zero values of u(^are gener- 
ated as follows. Each component of the continuous output vector is subtracted 
from a component of a constant threshold vector d G i?"" , and the resulting sig- 
nal e(^= d(8)y(^is the input to a zero detector which generates the components 
of the event signal t>(^according to 

, , _ f 1 if ei{t) = 0 and 3 A > 0 s.t. ej(t ® 5) >0 for 0 < 5 < A , . 

^ I 0 otherwise ^ ' 

Note that the zero detector is directional: an event is generated only when a 
component of y{t) approaches its threshold from below. Note also that a distinct 
event is generated when more than one threshold is hit at a particular instant. 
This reflects the fact that when multiple thresholds are encountered simultane- 
ously, the location of the continuous state is known more precisely than when 
any of the thresholds are encountered individually. 

In general; the analysis of the behavior of a hybrid system as described above 
is virtually intractable if the input signal u(^can switch at any instant. The 
scenario of interest is when the input condition signal u( ^changes values only in 
response to threshold events observed in v{^ This constraint on tt(^is enforced 
by the assumption that for all u 



h{u,0) = {u}, 

that is, the C/E system in Fig. 1 allows no transitions when the input event 
signal is zero. Thus, a signal pair is admissible for the hybrid system 

only if discontinuities in u(^occur only when u(^is non-zero. 

To assure u(^is piecewise constant with only a finite number of switches on 
any finite interval of time, we assume that u(-)^is nonexplosive (the term used in 
the Markov process literature [4]) or nonzeno (the term popular with computer 
scientists). That is, we assume that on any finite interval of time u(^is nonzero 
a only a finite number of, or zero, times. 

Following the development in [8], we define the discrete sequential behavior 
of the hybrid system described above as the language of the admissible sequences 
of pairs of discrete inputs and threshold events. In general this language, denoted 
Lh , contains strings of finite length and sequences of infinite length. Specifically, 
the sequential behavior corresponding to a signal pair (u(^u(^ is the sequence 
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of ordered pairs of the form where the times t* are the instants 

at which v{t) ^ 0, and u{t~) is the value of the discrete input that lead to the 
generation of the event v(ti). If this sequence is finite length, it is terminated by 
the pair (u{tf),v±), where tf is the time of the last threshold event in the signal 
t>(^and v± is a special symbol introduced to denote this null threshold event. 

Given a hybrid system, our interest is in creating a finite-state automaton 
that generates the language Lh- Since such an automaton may not exist, or 
may be too complex for the purpose at hand, our objective is to generate an 
approximating automaton A whose language La contains Lh ■ Such an automa- 
ton is useful either for verification of system properties or for discrete controller 
synthesis as described in the introduction. For controller synthesis, the nondeter- 
ministic C/E system in Fig. 1 represents the choices avaliable to the supervisor, 
where some restrictions may be imposed by logic already incorporated into the 
hybrid system cither physically or by lower-level discrete controllers. A super- 
visor designed to achieve specifications for the sequential behavior for La can 
then be converted to a supervisor which achieves the same specifications for the 
hybrid system as described in [3]. The remainder of this paper deals with the 
construction of approximating automata for hybrid systems of the form illustrat- 
ed in Fig. 1. Throughout the remaining sections the term hybrid system refers 
to this particular type of hybrid system. 



3 Mappings of Families of Continuous Trajectories 

The sequential behaviors of the hybrid systems are generated by continuous tra- 
jectories that evolve between threshold events. Therefore, computing the map- 
pings of families of trajectories from one threshold hypcrplanc to another is the 
key issue in the construction of approximating automata. This section introduces 
notation and definitions to deal with these mappings formally. 

Threshold events in a hybrid systems are generated by a set of threshold 
hyperplanes denoted Mi, ... , M„^. Threshold hyperplanc Mi is defined by row i 
of the output matrix C, and component i of the threshold vector d; that is. 

Mi = {x\cl X = di} 

Without lost of generality we assume ||ci|| = 1. We denote the negative, 
positive, and non-negative half-spaces determined by Cj and d.j by M^{cfx < 
di), M^{cjx > di),Mf^ {cjx -^di), respectively. 

Given a continuous state xq G i?"* and a set F we define the set of 

admissible continuous-state trajectories starting from xq with derivatives lying 
in F as 

X{xq, F) — { x(^j x(0) = Xo,x(t) E F for all t ^0, 
and i(^is piecewise continuous }. 

We let ZD{e{^ denote the event signal obtained when the (directional) zero 
detection operation is applied to the signal e(^ 
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The forward dynamic mapping is defined in terms of an initial continuous 
state xo, a discrete state u, a set T and an event vector v G V a.s 

■^u(xo ^T^v) = {xfGT\ 3x(^e T(xo,F„), and t/ > 0 s.t. 

Xf = x{tf), and for v{^= ZD{d ® Cx(^, 
v{tf) = V and v{t) = 0, VO < tf }. 

In words, A4u{xo ^ T t w), is the set of states in T that are reached by tra- 
jectories starting from xq, and when reached, the event v is generated. Note 
that the particular event generated when a trajectory reaches T depends on the 
direction (derivative) of the trajectory at that instant since the zero detector is 
directional. 

We generalize the above definition for a set of initial states S G i?"* as 

Mu{S ^ r T V) = U Muixo ^ T T V). 

xoES 

The backward dynamic mapping from T to S under u triggering ii is defined 

by 



Bu{T ^v~rS)^{xoeS I Muixo ^ T T u) ^ 0} 

In words, Bu{T 'I v ^ S) identifies the part of S from which T can be reached 
under u, triggering v as the first event along the trajectory. 

Finally, we define the null event states as the set of states from which there 
exists a trajectory which never triggers any event for a given discrete state u. 
Formally this set is given as 

J\fu = {xo G ! 3x(^e X{xo,Fu) s.t. 

v{^= ZD{d (8) Cx{^ and v{t) = 0,Vt ^0}. 

States in J\fu can lead to terminating behaviors, that is, to trajectories where no 
further threshold events are generated by the hybrid system. 

4 Approximating Automata 

In this section we summarize the method from [2] for constructing an automaton 
whose behaviors include all possible behaviors of the hybrid system. Let us first 
discuss the basic ideas of our approach. We start with a finite partition V of 
the threshold hyperplanes where each element of V is connected and completely 
contained in each element of some subset of the threshold hyperplanes. Given 
such a partition, we define an approximating automaton A as follows. Except 
for states labeled go and gj_, each state of A is labeled by a triple [P,u,v) G 
V -^V. There are two types of transitions in the automaton, the regular 
event transitions, each labeled by (u,u) G U ~^V, and the null event transitions, 
each labeled by {u,v±) G U A regular event transition from 

(P,u,v) to {P',u',v') is interpreted as follows. Given that a continuous state in 
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P has been reached under the discrete state u and the event v is triggered at P, 
there is possibly a continuous trajectory from a state in P to another state in 
P' under u' that triggers the event v' . The discrete state switching from u to u' 
must be consistent with the discrete transition function h, that is, u' G h{u, v). A 
null event transition {u',v±) from {P,u,v) to q± indicates that there is possibly 
a continuous trajectory under v! from a state in P that never triggers any event. 
The null state q± is a special state that we use to represent the case where no 
more threshold events occur, go is the initial state. 

To ensure that any subset S of a piece of an event hyperplane P ean generate 
any event that is possible for P, the partition P needs to be unambiguous, which 
is defined as follows. 

Definition 1. Let M = IJi=i n ^ partition P of M is unambiguous 
partition of the threshold hyperplanes if for all P G P, P is connected and 

either P -^Mk or P H Mfe = 0 V 1 (3) 

In the following we will also say any set T is unambiguous with respect 
to the set of hyperplanes if T satisfies the condition (3) with T 

replacing P. 

Definition 2. Given a hybrid system and an unambiguous partition P of the 
collection of threshold hyperplanes, we define the approximating automaton A 
corresponding to P as 



A = {Q,U,S,qo) 



where 

Q The set of the states Q = V -^U U {go, q±} 

S The alphabet is U ^{V ij {uj_}). 

5 The nondeterministic transition function 5 for A is defined according 
to the following rules 

1. (P, u, v) e (i(go, (w, u)) if ^ P { u) ^ 0 and u & Uq 

2. q± G 5{qo, {u, v±)) if Afu n Aq / 0 and u G Uq 

3. {P' ,u' ,v') G 5{{P,u,v),{u' ,v')) if A4it'(P P' J v') 0 and 

u' G h{u, v) 

4. q± G 5{{P, u, v), {u' , uu)) if Afu' H P 7^ 0 and v! G h{u, v) 
go is the initial state of A 

Letting La denote the collection of all possible finite-length and infinite- 
length sequences of input-event pairs that can be generated by the automaton 
A, it can be shown that Lh ~^La [2]. (In our context, a sequence is generated by 
an automaton A if and only if it is a sequence of labels on either an infinite-length 
directed path in the automaton starting from go, or the string of event labels 
on a finite-length path from go to gj_.) The extent to which A approximates H 
depends on the granularity of the partition P: if partition P' refines partition P 
with corresponding automata A' and A, then Lh ~^La' ~^La- 
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We now describe the procedure to obtain a finer partition V n+i from a given 
partition Vn- This procedure is shown in Fig. 2. Let Ajv be the automaton 
defined from the current partition Vn- For every hyperplane piece P G Vn-, we 
identify all the distinct evolutions of the continuous trajectories from P under 

{u ,v') 

different u’s. For each transition (P,u,v) {P' ,u' ,v'), we would like to 

isolate the subset of P corresponding to this transition. We use the backward 
mapping for this purpose. We can see that the subset we are looking for is 
exactly the backward mapping Bu'{P’ T v' P). Similarly, for each transition 

[P,u,v) g_i_^ the subset of P corresponding to this transition is simply 

Mu' n P. 

By computing the backward mapping and the null event set from all succes- 
sors of states containing P, we have a collection of subsets of P corresponding 
to different transitions in the automaton An- We then make a subpartition of P 
from these subsets. This process is represented by the routine disjoint in Fig. 
2. Given a collection of sets, disjoint returns a collection of disjoint sets whose 
union is the same as the union of the input sets. After we have proceeded through 
all P G Vn, all subpartitions of the sets P form the new partition Vn+i which 
is a refinement of . With the new partition, we can define a new automaton 
An+i- Therefore, L/ijv+i i-®-) each refinement step gives an approximat- 

ing automaton that is at least as good an approximation to the hybrid system 
as the one obtained in the previous step. Furthermore, it can be shown that if 
= Lan, then La^+i = Lh, i-e. if the refinement ever reaches a fixed 
point then the approximation, in terms of the sequences of input-event pairs, is 
exact [2]. 

5 Calculating Mappings and the Null Event Set 

The computations required to generate the approximating automata are similar 
to the computations employed in the tool HyTech [1]. In particular, the funda- 
mental operation is Reachp{X), which gives the set of continuous states reach- 
able from the set X iZ"* when the differential inclusion for the continuous- 
state trajectories is defined by the set F Formally, Reachp{X) is defined 

as 

Reachp{X) = {xf G | 3a;(^G X{xq,F) and tf ^0 s.t. x{tf) = Xf}. 

xoSX 

For a given set F defining 

(g)F = I s e F}, 

we have that Reach^p(X) corresponds to the time precondition operator defined 
in [1]. Reachp{X) corresponds to the time postcondition operator, which can be 
defined in a similar manner. Note that Reach^p(X) is the set of states from 
which the set X can be reached when the differential inclusion is defined by F 
since essentially runs the system backwards in time. 
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Vn+i= refine (Piv ={Pi,...,Ps} I Vn is a partition of M) 

{ 

s = \Vn\ 

P' = 0 

for t = 1 to s { 

A: = 0 

/* check regular event transitions */ 

{u' ,v') 

for each transition {Pj,u,v) {Pj,u' ,v') { 

.9 = T v' ^ PO 

/* add subset of Pi that maps under {u' .v') to Pj) */ 
if (5 ^ 0) { 
fc = fc + 1 
^ik ~ S 

} 

} 

/* check null event transitions */ 

/ t-t \ , 

for each transition (Pi,u,v) Q± { 

S = AfuC] Pi 

I* add subset of Pi that maps under u to q\_ */ 

if (S ^ 0) { 
fc = fc + 1 
P[k = S 

} 

} 

/* call disjoint to compute subpartition of Pi 
and form Vn+i from subpartitions of Pi’s */ 

V' =V' U disjoint({p'i, . . . ,p'fc}) 

} 

return V 



Fig. 2. Pseudocode for the partition refinement procedure. 
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The following definitions will also be useful later on in this section. Given an 
initial state xq and a set F { i?"® , the set of states reachable at time t from xq 
under differential inclusion F is defined as 

Reach i<'{t,xo) — {xf | 3a:({) e X{xq,F) such that x{t) — xj}. 

For V <E V and T unambiguous with respect to the set of threshold liyperplanes, 
define the set 



= n n (4) 

i S.t. TCM,,Vi = l i S.t. 7’CMi,C,=0 

In words, Rt^v is the set of states from which a state trajectory must arrive at 
T to generate the event v. 

Throughout this section, we assume that a target set T is unambiguous with 
respect to the set of threshold hyperplanes. 



5.1 Forward Mapping 

In this subsection, we present the method for computing the forward mapping 
set under a discrete state u given a set of states S called a source, a set of target 
states T, and the target event v to be triggered when T is encountered. The 
forward mapping A4u{S T ^ v) can be computed as follows. 

1. Rule out the part of S for which it is impossible to reach T and trigger v 
before any other event. The remaining part is the candidate initial set. 

CAN{S -^T-\v)^S Pi Mp P Mp 

i s.t. TCM,' i S.t. TcMi,v,=0 

2. Compute the set of via points in Rti;v to which any trajectory from S must 
pass through in order to reach T and trigger the event v before any other 
events. 

FVIAu{S ~^T]v) = ReachF,XCAN{S ^ T ^ v)) n Rt-^v P Mp 

i s.t. TCM + 

3. Compute the subset of T that is reachable from FVIAu{S ^ T ] v) 

FMAPu{S ~^T]v)= ReachFSFVIAu{S ~^T]v))C\T 

The following lemmas and propositions demonstrate that 

FMAPuiS ^T]v)= Mu[S ^ r T w). 

All proofs arc given in the Appendix. 
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Lemma 1. Given two states xq and Xf, if there exists an admissible trajec- 
tory Xi({) G X{xQ,Fy) reaching Xf at time t/, there exists another admissible 
trajectory X2({) G -Fu) which is a straight line connecting Xq and Xf. 



Lemma 2. If a trajectory a;({) G X{xQ,Fy) from xq G 5 to T triggers v before 
any other event, then xq G CAN{S T ] v). 



Proposition 1. Any trajectory x({) G X{xQ,Fy) from a state xq G 5 that 
reaches T and triggers v before any other event must pass through FVIAu{S 
T^v). 

Proposition 2. If x/ G Reachp^{FV I Au{S T v)) n T, then for any Xm G 
FVIAu{S ^ T v) such that x/ G Reachp^{{xm}) , there exists a trajectory 
x({) G X{xm,Fy.) that reaches x/, triggering v before any other event. 

Proposition 3. x/ G Mu{S T 1 v) <;==4> x/ G FMAPu{S ^ T 1 v). 

5.2 Backward Mapping 

The backward mapping under a discrete state u given a source S, target T and, 
a target event v can be computed in a manner similar to the forward mapping. 

Given event h G P. T unambiguous with respect to the threshold hyperplanes 
and a discrete state u, we define the backward mapping set Bu{T | v) as 

Bu{T I h) = { Xo I 3 x({) G X(xq, Fu),tf > 0 s.t. x(t/) G T, and for 

v{{) — ZD{d<Si Cx{{j),v{tf) — V, and x(t) = 0, VO { t<tf} 

The following steps can be used to compute Bu{S ^ T | v). 

1. Compute the set of via points in Rfii, from which T can be the first subset 
of threshold hyperplanes that is reached and thereby triggering v. 

BVIAu{T t v) = Reach^pST) n Rtm, H 

i s.t. TCM + 

2. Compute the set of states from which BVIA^iT | v) can be reached 

Reach^pSBVIAuiT ] h)) 

The above computation gives the set of candidate states from which T can 
be reached and v can be triggered. 

3. Restrict the above .set by some half spaces to yield the final result 

RMAP„(T t h) = Reach_F„(RR/A„(T T h)) p| M°+ p| M°+ 

i,TcMi,Vi—0 i,TcMi 
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The propositions that follow lead to our main result in this subsection that 
Bu{T t £i) = BMAPy_{T ] v), which is proved in Proposition 6. The backward 
mapping from T f v back to S under u is simply given by 

Proposition 4. Any trajectory x({) G X{xq,Fu) from a state xq ^ S that 
reaches T and triggers v before any other event must pass through BVIAu{T | 
v). 

Proposition 5. If Xm C BVIAu{T t v), then there exists a trajectory x({) G 
X{xm,Fu) that reaches some Xf G T, triggering v before any other event. 

Proposition 6. xq G S„(T | v) xq G BMAPu{T ] v) 

5.3 Null Event Set 

The following proposition gives the fundamental representation of the the set 
of reachable states for our hybrid systems and for linear hybrid automata. This 
result provides the basic method for computing the set of null event states. 

Proposition 7. Given a continuous state xq G i?"* and a discrete state u £ U, 

ReachF^{t,xo) = xq + Fut 

where 



xo + Fut = {x G R"'^\3w G Fu s.t. X = xo + wt} 

To compute A/”u we define for a state xq the set of indices for the threshold 
hyperplanes for which xq lies in the negative halfspace. This set of indices, and 
the intersection of the associated halfspaces is given as follows. 

Definition 3. Given a state xq we define Ix„ to be the set of indices i of the 
hyperplanes such that xq G M~ . We also define to be the intersection of all 
such ML . Formally we define, 



Ixo = e Af, } 

= n 

i6/xo 

The significance of Ixo is that threshold events will be necessarily generated 
from a trajectory starting from xq only for the threshold hyperplanes with indices 
in Ixo ■ The reason is that the directional nature of the zero detector implies that 
if i ^ Ixoj any trajectory crossing Mj from M~ must necessarily first cross Mi 
from M®^. However, if such a trajectory exists, there is another trajectory that 
will cross into Mi and just keep going. That is, it will not return and cross the 
threshold hyperplane Mi. These observations lead to the following proposition 
which completely characterizes Afu- 
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Proposition 8. xq G Nu Ixo — ^ or 3w € s.t. G cjw { 0 



To calculate A^„, we separate i?"* into 2"’^ regions as follows. Let I G {0, 1}"”, 
and define the set Rj to be 

B, = n "7 n "G- 

i:/j=0 i:L=l 



For each I G {0, 1}"“, we note that for all xq G Rj the set of indices is 
the same. Specifically, Ixu — {j\Ij — 0}. To decide if xq G A/’t, using Proposition 
8, we check whether there is a feasible vector w satisfying the following set of 
constraints 







■ 0 ■ 


[ J 


w { 





( 5 ) 



where for = {ii, . . . ,ik}, is defined as 



If (5) is feasible, then xq G AA„- Since the above check yields the same result 
for all xo G Rj, we have that Rj { Afu- After checking all possible I, we have 
that 



Nu= U Ri 

I s.t. (5) is feasible 

Note that there is no need to check for the case I = ln„xi- In this case 
Ixo — 0 nnd (5) is always feasible. 

6 Example 

In this section, we present a simple example to illustrate the construction of 
approximating automata for a verification problem, and to demonstrate the d- 
ifference between the approximating automaton approach and the fixed-point 
reachability computation approach for linear hybrid automata. Consider the lin- 
ear hybrid automaton shown in Figure 3a. We supply this automaton as an input 
to HyTech and compute the reachability region (for the continuous variables x 
and y) using the post operator, the result for the first three steps are depicted 
in Figure 3b. 

In this example, we would like to verify that starting from the set 5 = {0 { 
x { 1, y = 0} in location ui, the set T = {3 { x { 4, y = 1} is never reached. It 
is clear from Figure 3 that T cannot be reached from S. However, by iteratively 
applying the post operator in HyTech, the computation never converges as the 
reachability region continues to expand in the positive x direction. We now turn 
to the approximating automata approach. 
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(a) example linear hybrid automaton 

Fig. 3. 
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(b) reachable region 
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The above linear hybrid automaton can be easily converted into our model 
of hybrid systems introduced in Section 2 by defining the state variables x\ = x 
and X 2 — y with the output matrix C = [0 ( 01 ; 0 1] and the threshold vector 
d = [0 1]^. The threshold hyperplanes are <S>y = 0 and y = 1. The minus sign 
indicates that the hyperplane (B>y — 0 needs to be hit from above in order for 
an event to be generated. When the hyperplane = 0 and y = 1 are reached, 
the events [1 0]^ and [0 1]^ are generated, respectively. The transition function 
h{{ {) simply switches the differential inclusion to Fi = {i = y = 1 } on the event 
[1 0]^’ and to F 2 = {x & [3,4],y = (01} on the event [0 1]'^’. 

To define the first approximating automaton, we select the partition of the 
threshold hyperplanes shown in Figure 4a. We choose to include S and T in 
the partition because we would like to analyze their transitional behaviors. The 
automaton 0lo defined from this partition is also shown in Figure 4a. We drop 
the discrete state and threshold event from the automaton state labels as it is 
clear that any subset of {( 0 y = 0 } is labeled with u — 2 and u = [1 0 ]^ and 
any subset of {y = 1} is labeled with u = \ and u = [0 1]^. The labels on the 
transitions are also dropped because each of them can be inferred directly from 
the successor state. From this automaton, we see that T is reachable from S. 
Thus, we proceed with the automaton refinement procedure. 

To simplify our refinement in this example, we will only refine the sets in 
the partition corresponding to states that are reachable from S. As an exam- 
ple, consider the refinement of illustrated in Figure 4b. From the transi- 
tion diagram of 0lo, we see that B\ has multiple successors, namely Ai,S, 
and A. 2 . We find, through the use of backward mapping, the subsets of B\ 
from which Ai, S, and A 2 can be reached. Using our notation, we compute 
B 2 (Ai T [1 0]^ ^ Bi), B 2 {S T [1 0]^ ^ Fi), and 62(^2 T [1 0]"^ ^ Bi). These 
sets are indicated in Figure 4b. After making all of these sets disjoint, we break 
B\ into {Bn, Bi 2 , Bis, B 14 } as shown in Figure 4c. We proceed in the similar 
manner through all the subsets corresponding to the states that are reachable ! 
! from S in automaton Aq and obtain the next partition and the corresponding 
automaton Ai shown in Figure 4c. 

In the automaton Ai, T is still reachable from S. We refine the automaton 
once more and obtain the next partition and the automaton A 2 shown in Figure 
4d. In . 42 , T is no longer reachable from S and we have our verification result. 
Therefore, the property that T is not reachable in the hybrid system is veri- 
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(c) automaton Ai after first refinement (d) automaton A 2 after second refinement 

Fig. 4. 



fied after two iterations of the approximating automata construction algorithm 
whereas the HyTech hxed-point computation would never terminate. 



7 Discussion 

This paper develops a complete set of computations for generating approximat- 
ing automata for a particular class of hybrid systems characterized by linear 
constraints on the derivative of the continuous state vector. This demonstrates 
the possibility of carrying out the steps of the algorithm proposed in [2] (as 
well as the algorithm in [3]). The example in section 6 illustrates the use of ap- 
proximating automata for verification and compares the approach to hxed-point 
computations used in verihcation tools for hybrid systems. 

The hybrid systems considered herein are similar to the linear hybrid automa- 
ta. One difference between the models is the use of arbitrary discrete dynamics 
in the hybrid system in Fig. 1. In linear hybrid automata the discrete dynamics 
are dehned a priori as locations which determine the continuous dynamic modes. 
In the model described in section 2, the separation of the continuous and dis- 
crete dynamics allows for a more compact representation of problems in which 
the particular discrete undetermined. Such situations arise, for example, when 
the objective is to synthesize a supervisory controller to select the continuous 
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dynamics to satisfy a specification for the sequential behavior of the system as 
defined by the threshold events (see [3]). 

A more rigorous comparison of the model in section 2 and linear hybrid au- 
tomata should be made. It is evident an equivalent linear hybrid automaton can 
always be constructed when the discrete dynamics in Fig. 1 arc deterministic. 
On the other hand, the invariants of linear hybrid automata and nondeterminis- 
m in firing times for events defined by guards can be represented by thresholds 
and freedom in the linear dynamics in the model in Fig. 1. The model in Fig. 1 
does not admit jumps in the continuous state, which implies the linear hybrid 
automata are more general. (A more general model similar to Fig. 1 that admits 
jumps in the continuous state is described in [5].) In our opinion, the important 
difference in the models is the perspectives they represent. Linear hybrid au- 
tomata emphasize the discrete state aspects of the system, whereas the model 
in Fig. 1 emphasizes the continuous dynamics and a modular, signal-flow view 
of the world. Each perspective is useful for particular applications. 

Current research is focusing on algorithms for constructing approximating 
automata for hybrid systems with more general continuous dynamics. There 
are few applications where the continuous dynamics behave according to simple 
polytope constraints on the derivative of the continuous state vector. We are 
also investigating the application of approximating automata to the verification 
and synthesis of switching control laws in safety-critical systems. Results of this 
research will be reported in future papers. 

Appendix: Proofs 

Proof of Lemma 1 

Suppose ±i(t), 0 { t { tf is piecewise continuous with K pieces. Let to = 0 
and 0 < ti < . . . < t/c be the time at the end point of each piece. Note that 
tx — tf - For the piece, 1 { k { K, wc apply the Mean Value Theorem 
to obtain 



Xi(tfc) 0Xi(tfc-i) 
tfc 0 tk-i 



tk-l { tl{ tk 



Let Sk = and Xk = x\{tk)- Note that xx ~ x\{tx) = xi{tf) = Xf. We 

rewrite the above expression as 



Xk 0 Xk—\ — ^ki^tk 0 



It then follows that 

K K 

^ 0 Xk — \) — Xx 0 Xq — ^ ^ ^k(,tk 0 tk—l) 

k=l k=l 



Dividing by tx and letting = {tk 0 tk-i)/tx, wc have that 
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Xk 8) 3:^0 



K 

'y ^ <XkSk 

k=l 



Since ak is nonnegative and 



^ ^ tk ® tk-i 

k=i k=l 



f,K ® to 



the quantity s/ = «fcSfe is a convex combination of Sk- And because 

Sfc £ -F’ti VI { k { K, we have that s/ G Using the above results, we 
write 



Xk ® xo 
tx 



= s/ 



We construct the trajectory 



X2{t) = Xo + Sft 

Recalling that Xk — Xf and tx — tj, X 2 {t) is clearly a straight line connect- 
ing Xo and Xf. It also reaches Xf at tf. Finally, X2({) G A(xo,F„) because 
X2(0) = Xo, X2(t) is piecewise continuous (with only one continuous piece), 
and X 2 (t) — Sf G Vt { 0. 



Proof of Lemma 2 (by contradiction) 

Suppose that the trajectory x({) G A(xo,-F„), where xq G S', triggers v at 
Xf G T before any other event but xq ^ CAN{S ^ T | v). Then xq must 
fall into at least one of the following cases. 

Case 1 Xo ^ S. This contradicts the assumption that xq G S. 

Case 2 Xo G Ui s t tcm+ ■ Consider one threshold hyperplane Mi such 

that Xo G Mx and T { . Since T is not contained in Mj, — 0. Since 

Xo G and T { , any trajectory from xq to T must cross Mj from 

the negative side and trigger Vi before reaching T, a contradiction. 

Case 3 xq G |J^ g I; tcm v =o ■ Consider one threshold hyperplane Mj such 

that Xo G M~ and T { Mi, Vi = 0. In order to avoid triggering bj, x({) must 

approach xj from M®^. However, since xq G M~ , x(-j) has to cross Mi from 
the negative side before it can enter and trigger Dj, a contradiction. 



Proof of Proposition 1 (by contradiction) 

Suppose there is a trajectory x({) G X{xq,u) from a state xo G 5 which 
reaches T, triggers v before any other event but never enters FVIAu{S 
T t v). Let tf be time when v is triggered, i.e. x{tf) G T. Then x{t) must 
remain outside of FVIAu{S ^ T ] v) for all 0 { t { tf. It then follows that 
at any time 0 { t\ < tf, at least one of the following is true. 
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1 . x{ti) i Reachp^CANiS 

2. x{ti) € IJj g TCM+ 

3. x{ti) ^ 

Case 1 x{ti) ^ Reachp^{C AN {S ^ T | v)). This implies that xq ^ CAN{S 
T I v). Then, by Lemma 2, we have that x({) cannot reach T without 
triggering some other event before v, a contradiction. 

Case 2 x{t\) G Ui s t tcm' ■ In this case, there are some indices i such 

that T { and x{ti) G M~ . Let X be the set of all such i. By assumption, 
x{t) must eventually reach T at time tf > t\. Since T and x{ti) bracket Mi 
for all t G X, x{t) must have crossed all Mi from the negative side before 
it reaches T. Therefore, x{t) must have generated some other events before 
reaching T, a contradiction. 

Case 3 x{ti) ^ Rt'Iv From Case 1 and 2, we have seen that x{t) must remain 
in Reachpi^iS) Hi s t tcm+ all 0 { t < tf in order to have the 

possibility to satisfy our assumption. So we assume in this case that x(t) 
remains in Reachp^{S) Hist tcm+ at all time before reaching T. 

However, the definition of the event signal t>({) = ZD(d ® Co:({)) requires 
that x{t) enters Rt^v just before v is triggered. Thus, there exists a time 
t2 > t\ such that x{t2) G Rt]v and x{t2) G Reachp^{S) Hist tcm+ M^+. 
Clearly, x{t2) G FVIAu{S ^ X | u), a contradiction. 



Proof of Proposition 2 

Suppose Xf and x^ satisfy the above assumption. By Lemma 1, we can 
construct a straight line trajectory x{t) — Xm + st from Xm to Xf with 
x{tf) = Xf for some tf > 0 and s G Fu- We argue that this trajectory 
never generates any event before it reaches Xf. There are 4 types of event 
hyperplanes that can be encountered by x(j) during time 0 { t < tf. 

Case 1 Mi s.t. T { Mi and Vi = 1. From the definition of Rrfv, Xm G Al^f . 
Since x{t) is a line going from Xm G M~ to Xf G T { Mi, all of these Mi 
are first reached at from the negative side. Thus, none of these Vi are 
triggered during 0 { t < tf. 

Case 2 Mi s.t. T { Mi and Vi — 0 . From the definition of Rrfv, Xm G M^^. 
Since x{t) is a line going from x^n G to Xf € Mi, x{t) remains in 

for all 0 { t { tf. Thus, Vi cannot be triggered for all these Mi. 

Case 3 Mi s.t. T { M^. From the definition of FVIA^{S ^ T ] v), Xm G 
Since x{t) is a line going from xq G M^^ to x/ G M^ , x{t) remain in 
for all 0 { t { tf. Thus, Vi cannot be triggered for all these Mj. 

Case 4 Mi s.t. T { M~ . Xm may be on either side of Mi but Xf E M~ . Even 
if x{t) crosses Mi it does so from the positive side and, therefore, cannot 
trigger Vi. 

From Case 1, we see that all the events of interest in v is triggered for the 
first time at f/. From Cases 2, 3, and 4, we see that all the other events are 
not triggered at all. Thus, x{t) is one such trajectory that reaches T and 
triggers v before any other event. | 
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Proof of Proposition 3 

^ We prove this by contraposition. Suppose a:/ ^ F M APu{S —^T'l v). Then, 
Xf ^ Reachp^[FV I Au{S ^ T | v)) or Xf (jz T. 

Case 1 Xf then by definition Xf ^ A4„(S' T ] v). 

Case 2 Xf ^ Reachp^{FVIAu{S T 'I v)). Assume also for this case that 
Xf e T. By Proposition 1, any trajectory x({) G X{xq,Fu) from xq E S 
which reaches T and triggers v before any other event must pass through 
FVIAu{S ^ r t v). Thus, for Xf to belong to one such trajectory, it must be 
reachable from FVIAu{S -^T ] v). Thus, there is no admissible trajectory 
from S that reaches Xf and triggers v before any other event and we have 
that Xf ^ M.u{S ^ r t fi)- 

4= Suppose Xf £ FMAPu{S T '] v). Since Xf £ Reachp^{FV I Au{S 
T t v)) and FVIAuiS T ^ v) { Reachp^{CAN{S T ] v)), there 
exists a trajectory Xa({) £ X(xo,u) for some xq £ CAN{S ^ T | fi) to 
Xf G T via some Xm £ FVIAu{S ^ T | v). Without loss of generality, 
we assume that Xm is in Hist tcm^ because x{t) must eventually 
enter Hi s t rcM^ order to reach Xf. Since there exists an admissi- 

ble trajectory from xq to Xm, by Lemma 1, we can construct a trajectory 
xi{t) = Xq + Smt with xi{tm) = Xq + Smtm = which is a straight line 
connecting xq and Xm- There are 4 types of threshold hyperplanes that might 
be encountered by x\{t). 

Case 1 Mi such that T { . For each of these hyperplanes, both xq and x^, 

are in by the definition of CAN{S T ] v) and FVIAu[S -^T ] v). 
Thus, the whole line connecting them is also contained in for each Mi. 
Consequently, x\{t) cannot trigger the event Vi corresponding to any of these 
Mi for all 0 { t { tm- 

Case 2 Mi such that T { M£ . As discussed above, Xm £ for each of these 
Mi. Therefore, even if xq £ M^ for some Mi, Xi{t) crosses all of such Mi 
from the positive side. Thus, xi[t) cannot trigger the event Vi corresponding 
to any of these Mj for all 0 { t { tm- 

Case 3 Mi such that T { Mi,Vi = 0. From the definition of CAN{S ^ T '] v) 
and FVIAu{S T 'I v), for each of these Mi, both xq and Xm are in 
and so is the whole line connecting them. Thus, xi{t) cannot trigger the 
event Vi corresponding to any of these Mi for all 0 { t { tm- 

Case 4 Mi such that T { Mi,Vi = 1. From the definition of FVIAu{S ^ T | 
v), Xm £ M~ for all these hyperplanes. By the same argument as the one in 
Case 2, we have that x^(t) cannot trigger the event fij corresponding to any 
of these M* for all 0 { t { tm- 

From Case 1 to 4, we have that the trajectory x\ {t) cannot trigger any event 
up to time tm- 

Since Xm £ FVIAu{S T ] v) and Xf £ Reach p^{{xm}), there exists a 
trajectory X 2 {t) £ X{xm,u) which reaches Xf € T and triggers v before any 
other event by Proposition 2. Thus, we can construct a single trajectory that 
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triggers v before any other event from xq to x/ via Xm by combining xi(t) 
and X 2 (t) as shown below. 

x(t) = I 0{ t< trn 

The above result implies the left hand side of Proposition 3. 



Proof of Proposition 4 (by contradiction) 

Suppose there is a trajectory x(-{) C X{xq,Fu) which reaches T, triggers v 
before any other event, but never enters BVIAu{T \ ii). Let tf be the time 
when V is triggered. Then x{t) must remain outside of BVIAu{T ] v) for 
all 0 { t < tf. In other words, at any time 0 { ti < tf, at least one of the 
following is true. 

1. x{ti) ^ Reach^p,^{T) 

2. x{ti) E Ui, 7 'cM+ 

3. x(ti) ^ Brtv 

Case 1 x(ti) ^ Reach-p^{T). In this case x{t) cannot reach T for all t { ti, a 
contradiction. 

Case 2 x{ti) E IJ^ tcM+ ^bis case, we have a contradiction by the same 

argument as discussed in Case 2 of the proof of Proposition 1. 

Case 3 x{ti) ^ Rrtv- In tbis case, we have a contradiction by the same argu- 
ment as discussed in Case 3 of the proof of Proposition 1. 



Proof of Proposition 5 

Suppose Xm E BVIAy_{T | v). Since Xm must be in Reach-p^{T), Lemma 
1 implies that there exists a state Xf E T, a vector ®s E ®Fu, and a 
trajectory Xr{t) = Xf ® st such that x^itf) = Xm, which is a straight line 
connecting Xf and Xm- Since ®s E ®Fu s E we can construct a 
forward straight line trajectory x{t) = Xm + st from Xm to Xf with x[tf) = 
Xf. This trajectory never generates any event before it reaches x/ by the 
same argument discussed in the proof of Proposition 2. 



Proof of Proposition 6 

^ We prove this by contraposition. Suppose xq ^ BM APu{T | v), then .xq 
must fall into at least one of the following cases. 

Case 1 xo ^ Reach^p,^{BV I Au{T ] v)). In this case, any trajectory from xq 
can never reach a state in BVIAy_{T ] v). By Proposition 4, such trajectory 
cannot reach T and trigger v before any other event. 

Case 2 xq S IJ^ g ^ rcM+ ■ I'^ Ibis case, no trajectory from xq can reach T 
and trigger v before any other event (sec Case 2 in the proof of Lemma 2). 
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Case 3 a:o G IJi s t tcm e =o case, no trajectory from xq can reach 

T and trigger v before any other event (see Case 3 in the proof of Lemma 

2 ). 

<^= Suppose xo G BMAPu{T ] v). 

If xq is also in VIAu{T t v), the implication follows from Proposition 5. 
Now suppose Xo ^ VIAy_{T | v). Since xq G Reach-F^{VI A u{T | u)) and 
VIAu{T T t*) { Reach^p^{T), there exists a trajectory Xa({) £ X(xq,Fu) 
from Xo to Xf G T via Xm G VIA^{T | v). Without loss of generality, we 
assume that Xm is in PIiTcm^ ■ This is because Xa{{) must eventually 
enter P|^ tcm^ order to reach T. We can construct a two-piece linear 

trajectory from xo to Xm and from Xm to xj- and show that this trajec- 
tory triggers v before any other event at x/ as discussed in the Proof of 
Proposition 3. 



Proof of Proposition 7 

We prove that x G Reach p^{ti,xo) x € xq + Fut\ 

=> Suppose x G Reachp^{ti, xq). Then there exists a trajectory xi({) G T’(xo, u) 
such that X = xi(ti). By Lemma 1, there exists another admissible trajectory 
X 2 {t) — Xo + wt such that x = xq + wti for some w G Thus, we have 
that X G Xo + Futi- 

Suppose X G Xo + Fy^ti. Then x = xq + wti for some w G F,,. We construct 
a trajectory x{t) — xo + wt. Clearly, x({) G A’(xo,'u) and x(ti) = x, so we 
have that x G Reach p^{t\,xo). 



Proof of Proposition 8 

Wc prove this implication by contradiction. Suppose that Ixo ^ 0 and Vm; G 
F„, G Ix„ s.t. cjw > 0. For each w, we pick one such index and denote it 
by iyj. Since cJ^xq < di^ (by Definition 3) and cj^w > 0, there exists a time 
tw such that for t { cj^ (xo + wt) { di^ or, equivalently, xq +wt ^ Mg^ . 
Specifically, define 

C) cf Xo 

± ^ f'W 

^ 

Now, let 

* _ max 

It is clear that for all w G F„, c^^(xo + wt*) { di^. It follows that for all 
w E Fu, there exists an index i G Ixg, namely such that xq + wt* ^ AfG. 
This, in turns, implies that for all w G F„, xq + wt* ^ Sx„- By Proposition 
7 , we have that xq + F^t* = Reach p^[t* , xq ) { Sx„. Thus, for any trajectory 
x({) G A’(xo,tt), x{t*) G for some i G Ix^- But x(0) = xq G by 

Definition 3, so x({) must have crossed Mi from the negative side, triggering 
the event Xj, before or at t* . Therefore, we conclude that xq ^ Mu- 
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Case 1 If IxQ — 0 , then xq G for all i. We choose any vector w E Fy^ and 
construct an admissible trajectory x{t) = xq + wt. This trajectory can never 
generate any event. This is because even if x(-j) ever crosses any hyperplane 
Mi, it does so from the side and, therefore, cannot generate the event 

Case 2 If G s.t. Vi G Ixo, cjw { 0, we choose one such w and construct 
an admissible trajectory x[t) = xq + wt. We argue that this trajectory can 
never generate any event. For i ^ Jx„, xq G M^^ . The trajectory can never 
generate the event Vi by the same reasoning as in Case 1. For i G Ixoi 
cj Xq < di. Since x{t) = cf w { 0, cj x{t) < di for all t { 0. Consequently, 
x{t) can never generate the event Vi. Thus, for all i, x({) can never generate 

Vi- 

From Case 1 and Case 2, we conclude that xq G A/”„. 
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Abstract. Wc start from a basic and fruitful idea in current work on 
the formal analysis and verification of hybrid and real-time systems: the 
uniform representation of both sorts of state dynamics - both contin- 
uous evolution within a control mode, and the effect of discrete jumps 
between control modes - as abstract transition relations over a hybrid 
space A C Q X R", where Q is a finite set of control modes. The result- 
ing “machine” or transition system model is currently analyzed using the 
resources of concurrent and reactive systems theory and temporal logic 
verification, abstracted from their original setting of hnitc state spaces 
and purely discrete transitions. One such resource is the propositional 
pi-calculus'. a richly expressive formal logic of transition system model- 
s (of arbitrary cardinality), which subsumes virtually all temporal and 
modal logics. The key move here is to view the transition system models 
of hybrid automata not merely as some form of “discrete abstraction” , 
but rather as a skeleton which can be fleshed out by imbuing the state 
space with topological, metric tolerance or other structure. Drawing on 
the resources of modal logics, we give explicit symbolic representation 
to such structure in polymodal logics extending the modal /r-calculus. 
The result is a logical formalism in which we can directly and simply 
express continuity properties of transition relations and metric tolerance 
properties such as “being within distance e” of a set. Moreover, the log- 
ics have sound and complete deductive proof systems, so assumptions 
of continuity or tolerance can be used as hypotheses in deductive veri- 
fication. By also viewing transition relations in their equivalent form as 
set-valued functions, and drawing on the resources of set-valued analysis 
and dynamical systems theory, we open the way to a richer formal anal- 
ysis of robustness and stability for hybrid automata and related classes 
of systems. 



1 Introduction 

It is hardly controversial to claim that the /a- calculus is a formal logic of central 
import for the analysis and verification of hybrid automata and related classes of 
systems. The fundamental concepts of reachability and invariance are expressible 
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to Intelligent Systems”, grant no. DAA H04-96-1-0341. 



P. Antsaklis et al. (Eds.): Hybrid Systems V, LNCS 1567, pp. 38—69, 1999. 
© Springer- Verlag Berlin Heidelberg 1999 




On Hybrid Systems and the Modal ,u-calculus 



39 



in terms of fixed-points of operators mapping sets of states to sets of states, and 
are thus definable in the language of the /i-calculus. The iterative computation 
of the denotation of such fixed point formulas lies at the heart of symbolic model 
checking tools for hybrid and real-time systems such as HyTech [4], [19] and 
Kronos [13]. More generally, the propositional /i-calculus is well-recognized as a 
richly expressive logic over transition system models: the power of its fixed-point 
quantifiers are such that it subsumes virtually all temporal, modal and dynamic 
logics [15], [25]. 

However, the current practice, within the larger held of automated verihea- 
tion of (discrete) reactive systems as well as within the hybrid systems commu- 
nity, is to consider the //-calculus not as a working or usable logic but rather as a 
logic of the substratum. It provides a common “machine” language and seman- 
tics for veriheation by model checking, with user-input speciheations written in 
the more “natural” languages of temporal logics, and then translated into that 
of the //-calculus. 

This paper challenges that practice, and demonstrates that the propositional 
//-calculus and various of its modal logic extensions can provide both an expres- 
sively rich and “human readable” formalism for reasoning about properties of 
hybrid dynamical systems. 

We begin with the “machine” or transition system models of hybrid systems, 
in which both sorts of state transformation - continuous evolution within a 
control mode, and the effects of discrete jumps between control modes - are 
uniformly represented as abstract transition relations r { X { X over a hybrid 
state space X { Q { K", where Q is a finite set of control modes or discrete 
states. 

Formally, define a labeled transition system (LTS) (or generalized Kripke 
model) to be a structure 

( 1 ) 

where X ^ 0 is the state space (of arbitrary cardinality); for each transition 
label a G if, { X { X is a binary relation on X; and for each propositional 
constant (observation or event label) p G ||p||^ { X is a fixed subset of X. 

An LTS model is a clean and simple abstraction of a finite automaton. Such 
an 941 is an abstract machine over state space X, with input or action alphabet 

X and transition map 5 : X { X ^ 'Pi^) given by: x' G S(x,a) iff x > x'. 
It is additionally equipped with an observation alphabet <P, and an output map 
o : X ^ P(^) given by: o(x) = {p G | x G ||p||^}; sets of initial or final states 
can be identified by specific labels in (p. 

A (basic) hybrid automata Ti is typically represented by a graph of the form 
depicted in Figure 1. Hybrid automata and their associated LTS models are 
examined in more detail in Section 2; for now, we give a high-level description, 
based on Henzinger’s “time-abstract” transition system in [19] §1.2. 
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An LTS model DJI-h of a hybrid automaton H has a state space X { Q { M", 
with Q hnite. So states are pairs (q,x), where q E Q and x — (.ti, a:„) G K". 

For each q ^ Q, let Xg { K” be the projection of X under q. The transition 
alphabet E will include symbols such as for the relation of evolution (a 
“time-step” or “continuous transition”) within each discrete mode g G Q. In the 
basic case, such a relation is defined by: {q,x) {q,x') iff there is an integral 

curve along the flow 4>q connecting x G Xg to x' G Xg, and all points on the 
curve between x and x' lie inside the invariant set InVg { Xg. The transition 
alphabet will also include, for each edge (g, q') in the discrete transition graph 
G { Q { Q of 7d, a symbol Cg^gi for the controlled jump relation (a “step” or 
“discrete transition”) modeling the effect of making a controlled switch from 
mode q to mode q' . Such relations are standardly defined by: (q,x) {q',x') 

iff X G Grdq,qi , x' G InVqi, and x' G rg^q'{x), where rg,qi { Xq { Xg' is a reset 
relation for the real-valued coordinates, and the domain Grdg^gi { Xg is known 
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as the guard set of the discrete transition {q.q'). The alphabet of atomic 
propositions will include Init^ and InVg for q G Q, and Grdg g< for (g, q') G G. 

A trajectory of W is a finite or infinite sequence {Si, qi,"/i)i^j such that for 
each i G I\ the duration Si { 0; the curve 7 ^ : [0, ^ is such that 

(?i>7i(0)) <8^ (9i,7i(^)) for all t G {qi,qi+i) G G; and {q^,^i{Si)) "(gS' 

('7i+i>7i+i(0))- When I is finite, with largest element N, it is allowed that 
Sn — 00. When a hybrid automaton is thought of as a discrete controller in- 
teracting with a physical plant, the class of trajectories, so defined, are founded 
on implicit operational assumptions of continuous and perfect precision sensing, 
and instantaneous control switches ([19]). 

In the modal — as distinct from temporal — variant of the /r-calculus^, the 
propositional language (over an alphabet includes a dual pair of modal 

operators [a] and (a), for each transition label a G S. The (standard) relational 
Kripkc semantics of the labeled modalities are given by the universal and exis- 
tential pre-image operators of the corresponding relations r = a^. For relations 
r { X { Y, and sets A { Y, 

r(r)(A) A { X G X \ {Vy G Y)[ X y ^ y G A] } 

a{r){A) A { a; e A | (3y gY)[x y A y G A] } 

In the notation of [20], cr(r) = prc[r] and T(r) = pre[r]. The semantic readings 
of the modalities are forward-looking, and in temporal logics, they are known as 
relativized next operators: 

[a] (p “All a-successors satisfy ip” 

(a) p> “Some o-successor satisfies 

The temporal variant of the /r-calculus usually works with the global transition 
relation a^ (standardly assumed to be total) and the modal oper- 

ators arc replaced by global temporal “next” operators: VA or V{ , and 3 A or 

3{ . 

Sentences p of the /x-calculus denote sets of states ]| 93 || { A, and a sentence 

is true in 9Jl, written 9Jl N (/? , iff ||(y 9 ]| = A, or equivalently, ]|^¥^|| = 

0. The propositional connectives A and V are interpreted by set theoretic 
complement, intersection and union, and other connectives and constants defined 
in the usual way. In particular, ]|tt]|^ = A, and an implication p ^ if is true 
in 9Jl exactly when j|(^]| { ||^/;|j . As a point of contrast, in the language 

of linear temporal logic LTL, sentences denote sets of (finite or infinite) paths 
or trajectories of the LTS model, rather than sets of states. In the language 
of the branching temporal logic CTL*, there are two sorts of sentences: state 
sentences, true or false at states of the LTS model, and path sentences, true or 

^ The formal syntax and semantics of the jW-calculus are reviewed in detail in Section 
3 below. For an account of the modal and temporal flavors of the /r-calculus, see 
[38] §4.2. [15] is a good source for translations of various linear and branching time 
temporal logics into the /li-calculus. For background on modal logics, sec [9], [35]. 
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false of infinite paths through the model. An 3 or V path quantifier applied to 
a path sentence produces a state sentence, and such quantification is definable 
using the least and greatest fixed-point quantifiers of the ^-calculus. 

The principal advantage of working in the modal rather than temporal frame- 
work is that it gives a modular specification language for expressing properties 
of transition systems: we can describe and reason about each of the component 
transition relations of an LTS model, and how they are combined to form more 
complex transition relations. In particular, we can give a clean and modular 
formal description of classes of trajectories of the system. 

The modal sentences: 

■0 ^ [Cq,q']^ and ijj [e,](^ 

with the semantic readings “If ^ holds, then all -successors satisfy 99 ”, and 
likewise for e^, correspond precisely to Manna and Pnueli’s two types of (tem- 
poral logic) safety verification conditions for hybrid systems in [29] §4.1. Their 
notation is: and respectively, where r ranges over jump 

transitions and ^^cont” denotes the union of all the evolution relations. 

The modal sentence 

(®90 ) (‘^ 90 ,gi ) {®|J 1 ) (Cgi ,92 ) (®92 ) II II ll(®9fc-l ) ) (®9fc ) V’ (^) 

denotes the set of states (qo,x) from which some trajectory with discrete trace 
{qo,qi, ■ ■ ■ ,qk) reaches the set ||c^|| || X. Dually, the modal sentence 

[® 9 o] [c?o, 91] [®gi] 1^91,92] [^92] II II ll[® 9 fc-ll l^ 9 fc-l , 9 fe] [® 9 fc] V' ( 4 ) 

denotes the set of states from which all {qo,qi,... , (/fc)-trajectories reach the 
set ||(/ 7 || , upon the last jump and remain in ||(^|| throughout the last 

evolution Cq ^ . 

Defining e and c to denote the relational sum (union) of, respectively, the 
relations for the e^’s for g G Q, and the relations for the Cq^qds for {q.q') G G, 
the dynamics of the class of all hybrid trajectories with finite discrete traces are 
captured by the dual fixed-point definable modalities: 

(h) (yj A pZ. {e)(fiV {e){c)Z and [h] (/) A v Z . [e\Lp /\ [e\[c]Z (5) 

The sentence (h) ip “unwinds” to the infinite union of all sentences of the form 

(3) , and dually, [h] p corresponds to the intersection of all sentences of the form 

(4) . As a regular expression, we have h = (ec)*e = e(ce)* (so we are in fact 
working in the weaker propositional dynamic logic PDL, rather than the full p- 
calculus.) Semantically, (h) and [h] correspond to the dual pre-image operators 
of the reachability relation h of the system under the control of Tl; that is, 
{q,x) (5^ {q',x') iff some trajectory {5i,qi,^i)i^i with qo = q and 7 o( 0 ) = x 
passes through the point {q',x'). 

We now have the formal linguistic machinery to succinctly express various 
system specifications. The safety sentence 
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is true in the model SOT = exactly when every trajectory that starts in the 
set ||Init||^ always remains within ||99||^. More generally, we say a set ||i/?||^ is 
future-invariant under 7i exactly when the sentence > [h] y? is true in 941. We 
also have at our disposal (previously unutilized) deductive proof systems for the 
/r-calciilus, such as Kozen’s axiomatization [23], [5], [40], which is sound and 
complete over arbitrary LTS models. From the fixed-point rules of (given in 
Section 5), one readily derives an obvious invariance rule for hybrid trajectories: 

if ^ ip (p ^ [eg]y ip ^ [Cq,q']ip for qeQ, (g, q') £ G 
if [h](/p 

This is a simpler /x-calculus analog of the LTL invariance rule used in the veri- 
fication of safety properties for hybrid automata in [29] , [30] . 

To express liveness properties, we use modal analogs of the “box-diamond” 
construct in temporal logic. For example, the sentence 

[h](e)(c)(e)tt (8) 

is true in 9Jl exactly when every maximal trajectory from a state in Hoi’ll™ 
has an infinite discrete trace. This is so because [h] (e) (c) (e) tt denotes the set 
of states from which every trajectory with a finite discrete trace can be prop- 
erly extended. Similarly, the sentence p [h](e)(c)(e):/3 is true in 941 exactly 
when every trajectory from |[:/?|1^ returns to [[y?]]^ via a controlled jump in- 
finitely often. And [h](h)(^ denotes the set of states from which every hybrid 
trajectory eventually reaches Note that at this level of description, we 

cannot expressly rule out Zeno trajectories (^i, gi, 7i)iG/ such that I is infinite 
but < oo, but by considering variant evolution relations Cq dehned using 

a minimal time duration 5. we could. 

A clean /u-calculus definition of the higher-order modalities (h) and [h] also 
opens up new possibilities for aggregation in complex systems. We could model 
a complex system as a hybrid “meta-automaton”, where the dynamics at each 
discrete meta-mode p E P are given by the reachability relation hp of a (basic) 
hybrid automaton over state space Xp |] Qp || K", with switching relations 
from Xp to Xp> between automata, as illustrated in Figure 2. We now have the 
machinery with which to formally reason about the dynamics of such a creature. 

We also gain a clearer view of the enterprise of symbolic model checking for 
hybrid and real-time systems, as implemented in tools such as HyTech and 
Kronos. The basic task of such systems is to compute the reachable region of a 
hybrid dynamical system under the control of a given hybrid automaton 7t. As 
noted in the recent paper of Henzinger, Kupferman and Qadeer [20], to capture 
the notion “reachable from t^”, as distinct from “reaches p” , one needs in the 
semantics the post-image, rather than the pre-image, operator of a relation. The 

cleanest way to do it is to use the basic identity: post[r] = pre[r], where r is the 
relational converse or inverse of r, and to extend the ,u-calculus with a converse 
operation governed by the rule: 



{a)if —>■ p iff if ^ [a]p 



( 9 ) 
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Then the sentence 



(h) Init 



( 10 ) 



denotes the reachable region, where the post modalities (h) and [h] are defined 
as in (5), but substituting the converse relations. Symbolic model checking tools 

attempt to compute the value of || (h) Init ||®^ as a first-order formula in n + 1 
free variables {z,x\, ...,Xn), in the language £(M) of, say, the structure K = 
(K; <, +, ®, II, 0, 1, {q}q£Q) as the real closed field^ plus discrete constants. The 
procedure computes a sequence of first-order formulas yo, Xi; Xfc) which are 
translations of the /i-calculus formulas forming the approximation sequence for 

(h) Init, with the translation starting from the explicit hrst-order definitions of 
the set Init and the relations and Cq,q/ . The procedure terminates at stage 
fc + 1 if the formula: Xfc+i ^ Xfc provable in the first-order theory Th{M) 
of the relevant structure over M, in which case the reachable region is dehned 
by Xfc- The procedure is guaranteed to terminate when the model fOT = iXftji 
has a finite bisimulation quotient 91t~, where || is an equivalence relation on 
X II Q II M" which respects each of the transition relations Cq and Cq^q> and the 

^ The real closed field M admits elimination of quantifiers, so all first-order formulas 
in the language are provably equivalent in the theory Th(R) to a quantifier-free 
formula. The definable subsets of R" in R are the semi-algebraic sets: finite unions 
of sets defined by equalities and inequalities over polynomials / e R[Xi, ..., X„] [14]. 
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observation sets Initq, InVg, Grdq^qi. The recent work by Lafferriere, Pappas, 
Sastry and Yovine [27], [28], identifies a class of systems whose LTS models 
dJlu are first-order definable in an o-minimal structure M expanding the real- 
closed field. The finite cell decomposition property of such structures (together 
with a restriction on the form of the controlled jumps relations Cq,qi) is used 
to construct the finite bisimulation equivalence. (The theory of definable sets in 
o-minimal structures is developed in van den Dries’ monograph Tame Topology 
and O-minimal Structures [14].) 

The basic propositional modal /r-calculus can provide both a usable and a 
richly expressive formalism for reasoning about the abstract dynamics of hybrid 
systems. We want and need more. We want to be able to express in our logical 
formalisms what we mean by continuous and discrete dynamics, and hybrids 
of the two. We want to be able to formally express notions of imprecision or 
metric tolerance, such as the property of “being within distance e” of a set, for a 
particular e > 0. More generally, we want a logical formalism that supports not 
only the specification and verification of single properties, but the larger task of 
representing and building up a knowledge base of properties of a system, starting 
with structural properties assumed in the modeling, and then adding new facts 
as they are verified by either model-checking or deductive means. 

The remainder of this paper is an exploration of how the propositional modal 
/i-calculus can form a basis for a cohesive and expressively rich logical frame- 
work for the formal analysis of hybrid systems. In developing the logics, our key 
resources include: 

1. modal logics, considered as a general formalism for reasoning about binary 
relations and operators on sets ([9], [35], [38], [5]); and 

2. set-valued analysis and dynamical systems theory, brought into play by con- 
sidering transition relations r ]j X ]j X in their equivalent form as set-valued 
maps r ■. X X, i.e. functions r ■. X ^V{X) ([1], [6], [7]). 

In the course of this paper, it will be important to keep an eye on both the 
distinction and the interplay between: 

— the /r-calculus and various extensions as propositional modal logics (and thus 
ultimately monadic second-order logics [25]), in which formulas of the same 
formal language can be meaningfully interpreted in a variety of LTS models 
of any cardinality; in particular, in both continuum-sized models 971 and in 
finite quotients 97l~; and 

— the first-order languages £(K) and theories Th{M.) of specific structures K = 
(K; <,-|-,(8),j]0,l,...) over the reals, used in defining the components - the 
state space X, the transition relations and observation sets |]p||^ - of 
particular, albeit intended, LTS models 991. 

With regard to the latter, note that in the theory of o-minimal structures, rela- 
tions r : M"* M" go by the name of definable families (rj,)2;gRm ([14] §3.3). 
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To restate the point, the system description language is that of first-order 
logic, while the system specification language is that of propositional polymodal 
logic with fixed-point quantifiers. 

This paper is one instalhnent of a larger project. An analysis of the concept 
of bisimulation, and its relation to the algebraic semantics for the /x-calculus, is 
given in [11], and [12] gives the completeness of deductive proof systems for nor- 
mal polymodal extensions of the /i-calculus. Related logics and earlier versions 
of some of the ideas are found in [10]. 

The paper is organized as follows. Section 2 is a review and analysis of basic 
hybrid systems and their associated LTS models. Section 3 is a review of the 
syntax and LTS semantics of the modal /x-calculus. In Section 4, we flesh out the 
skeleton of an LTS model by imbuing the state space with topological and metric 
tolerance structure; we explore continuity and tolerance properties of relations r : 
X '^Y and applications to components of hybrid automata. Section 5 presents 
deductive proof systems for the new logics, extending Kozen’s axiomatization of 
L^. Section 6 is a brief discussion of ongoing research. 



2 Basic hybrid automata and associated LTS models 

First, a note on notation. For a set X, V{X) denotes the family of all subsets of 
X (a complete Boolean algebra). Following [6], the notation r ■. X Y means 
r II X II F is a relation, or equivalently, r : X ^ "P(X) is a set- valued map, with 
values r[x) || Y for x E X. The expressions: 

X y, {x,y)Er, y E r{x) and xry 

are synonymous. The domain of r : X F is defined by dom(r) A cr(r)(F), 

and the range ran(r) A cr(r)(X) = doni(r). Relational compositions r |[s of 
r : X Y and s : Y Z are read from left to right in sequential order, defined 
by: 



X z A E Y) x y and y z 

(cf. [1] where composition is written in the reverse order, as for functional com- 
position.) 

We base our discussion on a generalization of the systems considered in 
[27], [28], depicted in Figure 1. Figure 3 is an illustration. 

Definition 1. A (basic, evolution time- deterministic) hybrid system is a struc- 
ture 



where 



9:9' {Gr dq^qi }(,j g/)g(3) 
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— Q is a finite set of discrete states or control modes; 

— Gil Q II Q is the control graph of discrete transitions; 

— for each q € Q, 

I Xq II K” is the state space for mode q; 

II cfq : X, II M+ ^ X, is the continuous semi-flow of a vector field on Xq; 

I InVq II Xq is the set of invariant states for mode q, or the domain of 

permitted evolution within mode q; 

I Initq II InVq is the set of initial states for mode q (possibly empty); 

— for each discrete transition (g, q') G G, 

I Grdq^qi II Xq is the guard set for the jump from q to q' ; 

I Vq^q! : Xq Xgi is the reset relation; 
for X G Xq, Tq^qfix) || Xq' is the set of possible reassignment states after 
the jump from q to q' . 

The hybrid state space of the system H is the set 

^ = UeQMII 

To keep things simple, assume a hxed number n of real-valued coordinates, 
so Xq II K" for each q G Q. In [27], [28], the systems under consideration are 
simpler again in that they have constant reset relations Vq^qi — Grdq qi || Rstq^q>, 
with the constant set of reassignment states Rstq^q' || InVq>. 

The intention is that a hybrid system, so defined, is the semantic content 
of a hybrid automaton in the sense of Henzinger [19], Def. 1.1. For definiteness, 
we take a (basic, evolution time-deterministic) hybrid automata to be a hybrid 
system with a concrete syntactic description, namely: 

— the discrete structure is given by a finite graph (Q, G), where G ]] Q || Q; 

— each of the component sets Xq, Initq, InVq, Grdq^q> || K", semi-flows 

(fq : Xq II M+ ^ Xq, and reset relations rq,q' || Xq || Xq' have explicit 
hrst-order dehnitions in the language £(<, +, <8), |[ 0, 1, ...) of some specihed 
structure M over the reals. 

From [27], [28], we have reason to want such a structure K to be o-minimal. 

Operationally, a hybrid automaton H can be thought of as defining a non- 
deterministic hybrid control policy, partially defined on states (z,x) G X: 

if z = q and x G InVq 

then stay in discrete mode q and continue evolution according to </>,; 
if z = q and x G Grdq^q' for some (g, g') G G, 

then switch to discrete mode q' , re-initialize to some x' G rq^q'{x), 
and then evolve according to the flow fq' . 

The domain of definition of H is given by: 

dom(7f) = (U^eQ {?} II IriVq'j U (U(,,g')eG {«} II Grdq^q''j 




48 



J.M. Davoien 




If 2; = g and x G Grdq^g> for some (g, g') G G, then that discrete control switch 
is said to be enabled-, if (g, x) G dom(7f) but x ^ InVq, then some discrete 
control switch is said to be forced. It is generally assumed that rq^{x) || InVqi 
for all X G Grdq^qC, in words, Invqi is (forward) r,j^g/-invariant from Grdq^qi . In 
some expositions (e.g. [27]), it is required that Ti be total or non-blocking, which 
amounts to the assumption that dom(?-f) = X. 

In descriptions of the operation of a hybrid automaton and the ensuing class 
of trajectories of the system, it is generally assumed (e.g. [19]) that the state 
X — (xi,...,x„) G K" of the physical plant is being continuously sensed, with 
perfect precision, and that the action and effect of a discrete control switch is 
instantaneous. 

The accepted ([19], [27]) definition of the (“time-abstract”) transition system 
of a hybrid antomaton, with modified notation, is as follows. 

Definition 2. Given a hybrid system 7i, the LTS model DJln determined by TL 
has the following components: 

— the state space X = UggQ {g} jj Xq; 
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— for each discrete, state q € Q, the constrained evolution relation 
Cq : Xq Xq defined by: 

X x' = (3t G M^)[ x' = (f>q{x,t) A (VsG[0,t]) 4>q{x,s) £ InVq] 

— for each discrete transition (g, q') G G, the controlled jump relation 
Cq,q' ■ Xq Xq' defined by: 

X x' = X £ Grdq^q' A x' G InVq! A X x' 

— the observation sets Xq, Iriitg, Invq, Grdq q/ . 

We adopt the notational convention of identifying, when convenient, sets 
Aq II Xq and {g} || Aq \\ X; moreover, the relations Cq : Xq Xq and Cq^q' : 
Xq Xqi can be “lifted” to relations X X in the obvious way. 

From the dchnition of the evolution relation 6g, a desired property of the 
domain of evolution InVq is that it be convex with respect to the semi- flow fiq, 
in the sense that: 

if X G InVq and 4>q{x,t) G InVq for some t || 0, 
then 4>q{x, s) G InVq for all s G [0, t] 

So no curve segment of the semi-flow with both endpoints in InVq ever leaves 
InVq at an intermediate point. 

In the terminology of [1] Ch. 6, Definition 6.3, the (positive) orbit relation 
f : X X of a semi- flow f : X \\ K+ ^ X is defined by: 

X X = G X — 4>{x,t) (11) 

With respect to the orbit relation fq : Xq Xq of fig, the desired convexity 
property for InVq has the form: 

if xq,xi G InVq and xq x xi then x G InVq 

So when InVq if /g-convex, we have the decompositions 

e <7 = /? n {InVq II InVq) and Cq^q' = r,,,/ n [Grdq^q' || InVq') 

in which case we may as well assume the LTS model DJI-h includes the (uncon- 
strained) orbit relations fq and the uncontrolled reset relation r^.g/ . If we want 
to express properties which require both the orbit relation fq and its eonverse 

(convexity is one such), then we should include fq as a component of tMn as 
well (see also [20]). 

The modularity of the modal /it-calculus allows us to succinctly express not 
only desired properties - i.e. those to be verified, but also various of the structural 
properties of the LTS model dJlji that it will typically possess by assumption. In 
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a deductive framework, such sentences and sentence schemes (formulas with free 
propositional variables Z) provide an initial stock of facts known to be true in 
the model, and serve as hypotheses in application of inference rules when seeking 
to expand one’s stock of knowledge. 



[ 1 ] 

[ 2 ] 

[3] 

[4] 

[5] 

[ 6 ] 

[7] 

[ 8 ] 

[9] 

[ 10 ] 
[ 11 ] 
[ 12 ] 

[13] 

[14] 

[15] 

[16] 



(f,)Inv, A (fg)Invg ^ Inv, 

Initg ^ InVg 

Init ^ VgeQ 
Inv ^ VgeQ 

^ Inv,. 

(e,)Z ^ Invg A (fq){Z A Inv,j) 

(e,)Z ^ Inv, A (f,)(^ A Inv,) 

(c,,,/)Z ^ Grd,,,' A (r,,,')(Z A Inv,/) 

(f)^ - Vgeg (f,)^ 

Z ^ (f)Z 

(f,)(fg)Z^(f,)Z 

{e)Z ^ y qeQ 

(c)Z ^ V(g,g')gG’ (‘^9,9')^ 

(h)tt ^ Vg6Q Inv, V V(g,g')SG^I'd9.9' 



[1] says that /nw, is /,-convex. [2] is merely that Initq [| InVq. [3] and [4] define 
the global initial and invariant sets. [5] is the assumption that InVgi is (future) 
r,_g' -invariant from Grdq^q' . [6] says that every point in Grdq q/ has an r,_,/- 
successor; i.e. Grdq^q> || dom(r,,,'). [7] - [10] follow from the decompositions 
^q — fq^ (7nu, II InVq) and c,_,/ = r,^,/ fl {Grdq^q/ |[ InVq>). In particular, using 
the rule for converse (9) in Section 1 above, we have: 



V?->[e,](^ iff Inv, A (f,)((^ A Inv,) ^ 



(12) 



and 



f^lcq,q']f iff Inv,/ A (r,,,/)(((5 A Grdg,,/) ^ 



(13) 
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[11] defines / as the union of the orbit relations fg. From the zero semi-flow 
property, each fg is reflexive on its domain Xg, so / is reflexive (and total) 
on the whole space X, which is [12], From the sum semi-flow property, each 
fg is transitive; this is [13]. [14] and [15] are the definitions e = e, and 

c — '^{q,q')€G Cg^q’ ■ From [7], [14] and [12], it follows that: 

{Z A Inv) ^ (e)(Z A Inv) (14) 

that is, the relational sum e is reflexive on its domain. And from [7] and [13], wc 
get: 



{eg){eq)Z {eq)Z (15) 

which says each Cg is transitive. 

[16] defines the domain dom(77). The definitions of (h) and [h] in (5) above 
should also be added to the list. 

Using convexity assumption [1] and (12), the invariance assumption [5] and 
(13), and the invariance rule (7), it follows that Inv ^ [h] Inv will be true 
in i.e. the set Inv is future-invariant under H. More generally, whenever 

Inv ^ 9 ? is true in then Init ^ [h] 99 will be true, and thus on the 

current interpretation, |[93|| is safe under the action of 77, since no (perfect 
precision) hybrid trajectory starting in Init ever leaves Inv. So in this scenario, 
the situation of a controlled jump being forced - that is, (g, x) £ dom(77) but 
X (f InVq — can in fact never arise. Perfect precision trajectories start or land 
inside InVq, evolve continuously according to (fq, and then while the state is still 
inside InVg, or at worst on the (topological) boundary of InVq, a jump is made 
according to 

In some accounts of the LTS model of a hybrid automata (including that in 
[19]), the definition of the constrained evolution relation Cg is slightly weaker, 
with the requirement: V.s £ [0, t), (fq{x,s) £ InVg, so the end-point cf>q{x,t) 
need not lie in InVg. If InVq is closed (in the standard topology on Xg [| M"), 
then the continuity of <f>g : Xg || K+ ^ Xg entails that all such end-points will lie 
in InVq regardless, so the weakening makes no difference. In virtually all concrete 
examples of hybrid automata in the literature, the invariant sets InVq are closed. 

In Section 4, when we adjoin modalities corresponding to the interior and 
closure operators of a topology, we will be able to formally express properties 
such as being open, closed, or the topological boundary of a set. We will also be 
able to give formal expression to the assumption that the orbit relations fq are 
those of continuous semi-flows, and to consider consequences of continuity. 

We also clearly need to entertain the possibility that a physical realization of 
a hybrid automaton as a control policy might be less than perfect: sensors will 
be accurate only up to some level of precision; we should allow for delay between 
sensing the state and acting on that sensor reading in accordance with the control 
policy; and then there are margins of error in real-valued constants used in first- 
order definitions of the components of the model. In Section 4, we will consider 
alternative classes of hybrid trajectories by playing with the definitions of the 
fixed-point modalities (h) and [h] in an enriched modal language containing 
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modalities (e) and [e] interpreted by metric e-tolerance relations, for concrete 
values of e > 0. 



3 Syntax and LTS semantics of the modal -calculus 

The /x-calculus originated in the late 1960’s (Scott and de Bakker) as a formal 
logic of digital programs, the input-output behavior of an atomic program being 
represented as a binary transition relation on (discrete) states. Contemporary 
introductions to the /e-calculus can be found in [38], [15]. In this section, we 
review the syntax and semantics over LTS models of the propositional modal 
/Lt-calculus. 

Definition 3. A modal signature is a pair (^, S), where ^ is a set of proposi- 
tional constants and S is a set of transition labels. Let PVar denote a fixed set 
of propositional (second-order or set-valued) variables. The collection , S) 
of formulas of the propositional modal pL-calculus is generated by the grammar: 

(/?::= ff ] p \ Z \ \ ifiW (p2 \ {a)ip \ pZ.ip 

for propositional constants p ^ <P, propositional variables Z G PVar, and tran- 
sition labels a € S, and with the proviso that in pZ.p, the variable Z occur 
positively, t.e. each occurrence of Z in y is within the scope of an even number 
of negations. 

The other (classical) propositional connectives, modalities and greatest fixed 
point quantifier are defined in the usual way: 

tt V -iff /\ ip2 Az -i(-'yi V -i(/72) 

ipi (f2 = V Ti ^ T2 = (yi ^ T 2 ) A {ip2 yi) 

[a]if = -i(a)-iy vZ.p = -^/jZ.^ip[Z := -^Z\ 

An occurrence of a variable Z G PVar in a formula that is within the scope of 
a pZ is called bound, otherwise it is free (as in first-order logic). Let S^{<T,S) 
denote the set of all sentences, or closed formulas of .7yj(^, A), i.e. those without 
any free variables, and let J-[d>,S) and 5(<Z>, A) denote, respectively, the set of 
all purely modal formulas and sentences, i.e. those containing no fixed point 
quantifiers, and in case of sentences, no variables Z . 

For formulas p,ip & ^)i let Lp[Z := if] denote the result substituting if 

for all free occurrences of Z . By renaming bound variables in p if necessary, we 
can assume such substitutions do not result in the unintended capture of free 
variables. 

Definition 4. Given an LTS 911 = {X, {a^}ass, {\\p\f^}p€’p) of modal signa- 
ture {<P, S), a (propositional, or second-order) variable assignment in 911 is any 
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map ^ : PVar ^ V{X). Each such assignment ^ uniquely extends to a denotation 



f-.E,{<l>,X) 




V{X) as follows: 




llffllc 




0 




II ii®t 

l|p|le 


= 


ibir 


for p E <F 




= 




for Z G PVar 


Ihvllf 




X® ll^llf 




Wfi V P2\\^f 


= 


IIVl|irU||:^2||f 




ll(«)‘/5||f 




(llvllf ) 


for a E X 


IlM^-Vllf 




n{A G PiX) 1 


A} 



where the pre-image operator cr(a™) is defined as in (2) above, and for sets 
A G V{X), the variant assignment : PVar ^ ^{X) is given by: 

f{A/Z){W) ^f{W) if W ^ Z, and f{AfZ){W) ^ A if W ^ Z. 

For formulas 99 G X) and assignments f : PVar ^ 'P(X) in 971, we say: 

— (f is true at state x in (971,^), written: 1= gs, iff x G 

— ip is true in (991,^), written: 991, t= p, iff ||</9||^ = X; i.e. p is true at 
all states x in (991,,^); and 

— p is true in 99t, written: iXfl\= p, iff p is true in (991,.^) for all assignments 
f in 99T. 

For sentences p G S^{d?,S), the denotation ||!, 9 ||^ is independent of the 

variable assignment f, and is written ||(/ 9 ||®^. So 991 1= (^5 iff 991,4 F for 
assignment f. 

Given a model 991 and variable assignment 4, each formula p G X) and 

each variable Z G PVar free in p, together determine an operator on sets 
pf^z ■ given by: 

("4) = llp||{(A/z) (f®) 

The variant assignment construct corresponds to substitution: for all formulas 
if G Tff^,X), 

Wfz)m\f) = Mz-.= mf ( 17 ) 

When the variable Z occurs positively within p, so pZ.p G XffF, V), the oper- 
ator P^z i® II -monotone: 
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for F — The clause in Definition 4 for /r- formulas says that is 

the II -least pre-fixed-point of the monotone operator complete lattice 

V{X). So by the Tarski-Knaster fixed-point theorem, \\pbZ.Lp\^ must also be the 
I -least fixed-point of i®- 

\\pZ.^\\f = f]{AeV{X)\Mf^^/z)=A} 



In the standard set-theoretic semantics for the /Lt-calculus, as presented here 
and given in [23], [38], [40], [15], the propositional variables Z range over the 
full power-set (and complete Boolean algebra) V{X) - that is, all subsets of X. 
An alternative, developed by Kwiatkowska and colleagues [5], [8], is an algebraic 
semantics in which the range of propositional variables is restricted to a sub- 
family A II V{X). This work has roots in a number of classic studies from 
the 1950’s, notably that of Henkin [18] on completeness of higher-order logic; of 
Jonsson and Tarski [26] on Boolean algebras with operators; and that of Rasiowa 
and Sikorski [36] on algebraic logic. 

Definition 5. ([5], [8]). Given an LTS model 911, a family of sets y4 || V{X) 
is said to be a modal algebra for 911, and the pair (911, A) is known as a modal 
frame, when each of the following holds: 

1. A contains each of the observation sets ||p||^, for p G 

2. A is a Boolean algebra under the finitary set-theoretic operations; and 

3. A is closed under each of the pre-image operators a(a^) and for 

a G A, 

For purely modal formulas p> G F{d>,X), the clauses in the inductive defi- 
nition of the denotation ||<^||^ || X with respect to a modal frame {^fft,A) are 

identical to those in Definition 4 for ||</5]|™, with the proviso that variable as- 
signments ^ are restricted to A, i.e. ^ : PVar ^ A. 

A formula ip is true in the frame (911,^1), written (911, N iff ||<y9||^ = X 
for all assignments ^ in A. 

An LTS model 911 is identified with the modal frame (911, P(X)). 

Modal algebras A || 'P{X) need not be complete as lattices, so unlike V{X), 
we have no guarantee that the set being the || -least pre- fixed-point of pi'^z 
fact exists in A; when it docs, it is the least fixed-point in A of ‘P'^z^ ^ variant 

of the argument in the Tarski-Knaster fixed-point theorem. 

Definition 6. ([5], [8]). A modal algebra A || P{X) is called a modal ^u-algebra, 
and the pair (911, A) called a modal /r-frame, if for each formula pZ.p G S) 

the infinitary meet or infirnurn of the family in A of pre-fixed-points of pf'z 

A{^ G A I ||<(5||^(yj/;^) II A } 

exists in A, in which case ||/xZ.(/5||^ is that set. 
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In general, the denotations ||93j||^ and ||v?||^ part company on /i-formulas, 
since the smallest of all sets A G V{X) such that a condition holds will be 
contained in the smallest of all sets A £ A for which the same condition holds. 
In [11], we identify conditions under which a modal /x-framc (911, A) is in semantic 
agreement with 991, i.e. for all /r-formulas ip G i7), ||</?||^ = llv^ll^ for eiH 

assignments ^ restricted to A. The smallest ;U-algebra for an LTS fOI is the 
countable algebra 

s^={\\p\r\pGS,{^,x)} 

of denotations of jU-sentences in 991. It is readily verified that is in semantic 
agreement 991. 

From the purely modal clauses in Definition 4, together with the dehnitions 
of the pre- image operators in (2), it follows that if the state space, transition 
relations and observation sets of an LTS model 991 are all first-order definable 
in some structure, then for all modal sentences p G the denotation 

|(y9||^ II X is first-order definable. Otherwise put, the countable algebra 

= { llc^ir |(/,G5(^,i:)} 

of denotations in 991 of purely modal sentences, has a finitary syntactic repre- 
sentation as a family of first-order formulas; a family finitely generated by the 
explicit first-order definitions of the components of 991, under the straight-forward 
translation of modal sentences based on the definitions (2) and the (classical) 
meaning of the Boolean connectives. Of course, an optimal situation is when the 
first-order structure admits quantifier- elimination, as then the naive translation 
of a modal sentence can be reduced to a quantifier-free formula, and so the al- 
gebra will have a simpler and more tractable representation. Such algebras 
are the semantic content of Henzinger’s notion of a symbolic execution theory in 
[19] §3.1. 

Returning to the standard set-theoretic semantics, the completeness of V{X) 
as lattice ensures that the set ][juZ.;^[|^ has an equivalent characterization (by 
the Park-Hitchcock fixed-point theorem) as the union of an || -increasing se- 
quence of approximations: 



where 



\pZ.p\\f- U 

a<Ord(m) 






= 0 



N ot o ot 
f,a+i = nz 



iiOT 



= u 

a<r) 



\M 

I 



OT 



for limit ordinals r] 
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and Ord{DJl) < for k — Card{X), is the closure ordinal of 971. The sets 

are /^-approximations of ||/rZ.</;||^. Likewise, the denotation of vZ.(p can be 
represented as the intersection of an || -decreasing sequence of i^-approximations. 

In the general case, over LTS models 971 of arbitrary cardinality, approxima- 
tion sequences for the denotation of fixed-point formulas proceed through trans- 
finite ordinals; when X has the cardinality of the continuum, Ord{1Xft) could be 
much longer than we care to deal with. 

When the operator corresponding to the body of a /t-formula pZ.p is 
u>- chain- additive, that is, for F — 

( U I ^ U where An || A„+i for all n < w 

\n<Lo / n<ut 

then the ordinal of convergence for ||/tZ.(/ 7 ||^ is at worst uj. In this case, we have 
a sequence of approximation formulas 

A g and 93 "+^ = p[Z p^] for n < w (18) 

and 

\\pZ.p\\f= U ||^"l|f 

n<o; 



since terms “order-continuous” and “continuous from be- 

low” are also used instead of cn-chain-additive, since such an F : V{X) F{X) 
is a continuous function with respect to the Scott topology on the complete par- 
tial order (V{X), || ). We adapt the terminology of Jonsson and Tarski [26] on 
Boolean algebras with operators, since we are interested in other meanings of 
“continuous”. Dually, when p^z oJ- chain-multiplicative, the ordinal of conver- 
gence for \\nZ.p\\^ is at worst uj, and the sequence of approximation formulas 
starts at tt and decreases. 

In particular, the semantic operator corresponding to the body of (h)(/) (or 
(h)(/j), as defined in (5), for sentences p, is: 

A a{e){\\p\f^) U a{ec){A) 

Since the 3-pre-image of any relation is completely additive, i.e. distributes over 
arbitrary unions, it follows that ||(h)i/j||^ is the union of the denotations of the 
approximation sequence 

ff, {e)p, {e)pW {e){c){e)p, {e)p W {e){c){e)p V {e){c){e){c){e)p, ... 

Dually, the semantic operator corresponding to [h] is completely multiplicative. 

When II is a bisimulation equivalence on 971 - that is, an equivalence relation 
on X which respects the transition relations and the observation sets ||p||^ 




On Hybrid Systems and the Modal ,u-calculus 



57 



in a suitable sense^ - then the fundamental property of truth-preservation is as 
follows: for all sentences Lp G S^{$. S) and all x, y G X, 

X II y ^ [xe\\ip\f^ ^ ye\\ip\f^] (19) 

It follows that if II is a bisimulation equivalence of finite index N, then the 
denotation ||y?||^ of each sentence is a finite union of equivalence classes under 
I . Hence for sentences iiZ.(p and nZ.ip, the ordinal of convergence for \\iiZ.Lp\\ 
and WvZ.ipW is bounded by N. In this case, the finite quotient LTS 911“ is a 
finite simulacrum, and finite automaton representation, of the original system 
911. If such is the case, the countable ;U-algebra is in fact a finite algebra, 
and the atoms of the algebra are the equivalence classes under || . The familiar 
bisimulation algorithm ([19] §3.1; [27] §2) can be reinterpreted algebraically as 
the construction of a sequence of algebras 5® for k < uj, where 

sr^{h\r\pes,i^,s)} 

is the finite Boolean algebra of denotations of modal sentences of modal degree ]] 
k. The modal degree measures depth of nesting of modal operators; for example, 
for hybrid trajectory formulas of the form (3), the degree is 2n + 1, where n 
is the length of the discrete trace. It follows that 5^]^ is the smallest Boolean 
algebra generated by U {a{a^){A) \ A G >5®}. The algorithm terminates at 
stage fc + 1 if S'^-y — S^, in which case the equivalence relation: 

^ II s™ y ~ ^ A y G H ] 

is a finite bisimulation equivalence whose equivalence classes are atoms of the 
algebra 5®, and = S^. 



4 Adding topological and metric tolerance strnctnre 

Within modal logic, there is a well-known way of representing a topology T on 
the state space X of an LTS or Kripke model. From McKinsey and Tarski’s 
work in the 1940’s ([31], [32], [36]), the axioms for the box □ modality of the 
modal logic S4 correspond exactly to those of the Kuratowski axioms for the 
topological interior operator intr, and dually, the S4 diamond <> corresponds 
to topological closure cIt- S4 is a well-studied modal logic, and is of particu- 
lar interest in virtue of the 1933 Gddel translation of Intuitionistic logic into 
(classical) S4. The relational Kripke semantics for S4 is in terms of pre-orders: 

® The concept is not formally defined here. An analysis of the concept of bisimulation 
is given in [11]. See also the handbook article [38] §5.3, where it is noted that if 
one wants to preserve the truth of sentences containing the converse operation, then 
the notion of bisimulation must be strengthened so as to include respect for the 
converses of the a®*. 
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reflexive and transitive relations =^|| X || X, and can be shown to be a special 
case of the topological semantics via AlexandrojJ topologies, which are in one-one 
correspondence with pre-orders (see [11]). For background on general topology, 
sec [33], [24], 

Let S) denote the collection of formulas defined as in Definition 3 

with an additional clause for a plain □ modality, with analogous notation for the 
collection of sentences, and the purely modal fragments. The diamond is defined 
by the usual negation (de Morgan) duality: Op = 

Definition 7. IfdJt= {X,T, [aP^}a^Ei {||p||^}pe<f) a topologized LTS mod- 
el then the additional clauses to be added to Definition 4 for the semantics of 
formulas p € F’) are: 

|1□93||^ = -intr and \\Op\\^ ^ dr {\\p\\f''^ 

In the enriched language, we can simply express topological properties of sets 
of states. For example, a set |]9?|]^ ]] X is, respectively, open, closed, dense or 
nowhere dense (empty interior), with respect to T, exactly when the sentences 
p Op p. Op, or O^p are true in 911. The topological boundary 

of is denoted by the sentence Op A —IDp (and boundary sets are always 

nowhere dense). 

Note that if j| K" is first-order definable in an o-minimal structure K, 
T is the subspace topology on X inherited from the standard metric topology 
on K" (derived from the order < on M), and ^ jj X is definable, then intr{A) 
and clr{A) are also definable ([14], Lemma 3.4). Thus if the components of a 
topologized model 911 are definable in M, then the topological modal algebra 

S^ = {\\p\r\peSai^,X)} 

of denotations of modal sentences including □ is also definable. From the perspec- 
tive of o-minimality, observe that the cells of a cell decomposition of a definable 
X I] K” are either open in K”, or else are boundary sets ([14], Proposition 2.5) 
- properties expressible in the enriched modal language. 

Note that if we want a bisimulation to be truth-preserving with respect to 
sentences p E X), then it must also respect the topology T. For equiv- 

alence relations |j , this amounts to the requirement that for each equivalence 
class B under j] , the closure clr{B) must be a union of equivalence classes, thus 
either intr{B) — B oi intr{B) — 0; in brief, the equivalence classes B are 
“cell-like” . 



OK, so we’ve formally got topologies in the picture, so we should be able to 
express some notion of continuity. A sticking point is that the standard notion of 
continuity is for functions, not relations. In purely topological terms, a. function 
f : (X,T) (K, iS) is a continuous iff for every open set U in Y, the inverse- 

image f~^{U) is open in X. The relevant notions for relations r : [X, T) {Y, S) 
were introduced by Kuratowski and Bouligand in the 1930’s, and replace the 
functional inverse-image with the relational V- and 3-prc-image operators. 
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Definition 8. ([6] §1.4; [1]"^ Ch. 7; [24] §18.) A relation r : 

— upper semi-continuous (u.s.c.) iff for every open set U in Y , the^ -pre-image 
T(r)(U) is open in X ; 

— lower semi-continuous (l.s.c.) iff for every open set U in Y , the 3-pre-image 
a(r)[U) IS open in X ; 

— continuous iff it is both u.s.c. and l.s.c.. 

When r : {X,T) {YjS) is in fact a (single- valued) function, each of the 
semi-continuity properties is equivalent to functional continuity, since in that 
case, the two relational pre-image operators collapse to the familiar inverse- 
image operator: a{r) = rfr) = r^^. Logics of continuous functions arc developed 
in [10]. 

The two semi-continuity properties are simply expressible in the language of 
the topological /r-calculus by the formulas (sentence schemes): 

[a]DZ ^ n[a]Z and (a)DZ D(a)Z (20) 

In dual form, upper semi-continuity can be read as preservation of closed sets 
by the familiar 3-pre-image cr(r) = Pre(r): 

0{a)Z-> {a)OZ 

From these simple characterizations of the semi-continuity properties, it follows 
purely formally that each of the properties is inherited under finite relation- 
al compositions and finite relational unions (sums). Inheritance of continuity 
properties under inhnitary hxed-point quantihcation is a topic of continuing in- 
vestigation. 

So far, the discussion of continuity is still rather formal, and a tad insubstan- 
tial. But in the case of compact metric spaces, we get to see some meat on the 
bones. 

Proposition 1. ([1] Ch.7, Proposition II) For relations r : X Y where X 
and Y are compact metric spaces and the direct image r{x) || Y for each x E X 
is closed, the following are equivalent: 

1. r is u.s.c.; 

2. for all X € X and all e > 0, there ts a d > Q such that for all x' G X and 

y' e 

dx{x,x') < S and x' (8^ y' {3y G Y)[x y and dY{y, y') < e ] 

3. as a subset of X || Y, (the graph of) r is closed; 

^ Note that in [6], [7], Aubin uses the terms “core” and “inverse-image” instead of 
universal and existential pre-image, while in [1], Akin uses has but has neither names 
nor notation for the pre-image operators. 
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4- r : Y X is u.s.c. 

The following are also equivalent: 

1. r IS I.S.C.; 

2. for all X € X and all e > 0, there is a 6 > 0 such that for all x' G X and 

y&Y, 

dx{x,x)<5 and x y ^ {3y' GY)[x' y' and dy(y, y')<e] 




Fig. 4. The u.s.c. property in the compact metric setting. 



The metric u.s.c. property says that if an input x' is within S of x, then every 
point y' in the output or image r(x') is contained within an e “ball” or “tube” 
around r{x). For the orbit relation f \ X X oi a. semi- flow (f> ■. X \\ ^ X 

(defined in (11)), where f{x) — {4>{x,t) \ t G M+} is the positive trajectory from 
X, the picture really is that of an e-tube: if dx{x,x') < S then the trajectory 
f{x') lies inside an e-tube around the trajectory /(x), as illustrated in Figure 
4. The idea is certainly reminiscent of the “tube neighborhoods” in the work of 
Gupta, Henzinger and Jagadeesan [17] on robust timed automata; the interest 
in that paper is on metrics on trajectories r G (^ || K^®)*, where is a finite 
alphabet of event names. 

When X is a compact metric space, (j) : X \\ ^ X is a continuous semi- 

flow, and T || K+ is compact, the restricted orbit relation : X X given by 
/^(x) = {(j>{x, t) \ t G T} has a closed graph and hence is u.s.c. ([1], Ch. 6). This 
leads to the following result on continuity properties of both sort of transition 
relations in an LTS model of a hybrid automaton. 

Proposition 2. Let dJl-H be the LTS model of a hybrid automaton, as in Defini- 
tion 2. Assume that each Xq || K" is compact in the standard topology onM". Let 
Tq be the subspace topology on Xq, and assume the semi-flow cfq : Xq || M+ ^ Xq 
is continuous. 

1. If InVq is closed in Tq, and time-bounded under fq, in the sense that there 
is a tq > 0 such that for all x G InVq and all t > tq, (pq{x,t) ^ fq, 
then the relation e, : Xq Xq defined by Cq = fq H {Tnvq || Tnvq) is u.s.c.. 
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2. If Grdq^qi T Xq and InVq> T Xqi are both closed, in Tq and Tq/ respectively, 

and the graph of rq q/ : Xq Xq/ is closed, 

then the relation Cq qi : Xq Xqi defined by Cq^qi = rq^qi D {Grdq qi T InVqi) 

is U.S.C.. 

The point is that the u.s.c. property is sufficiently attractive that we may 
wish it to be the case that all our transition relations possess it. From our 
observations above, all finite compositions and unions of the Cq and Cq^q' will be 
u.s.c. if the 6q and Cq^q> are u.s.c.. Note also that for the constant jump relations 
Cq,q' = Grdq^qf T Rstq^qf of [27], Cq_q' is U.S.C. wheU both Grdq^q' and Rstq^qf are 
closed. 

When the relations Cq ; Xq Xq and Cq^q' : Xq Xq' are lifted to relations 
X X, the issue arises as to what is the appropriate topology on the hybrid 
state space X T QTM” ? Taking the Xq equipped with their standard topology 
from K", the question then becomes: what topology Tq on the finite discrete state 
space Q'! One reasonable choice is that Q really is discrete and has no topological 
structure, which amounts to taking Tq to be the discrete topology. Then the 
lifted relations will be u.s.c. or l.s.c. whenever their unlifted counterparts are. 
An alternative reasonable choice is to consider Q as structured by the control 
graph G T QT Q, so take Tq — Tq to be the (Alexandroff) topology determined 
by the reflexive-transitive closure of G. The open (closed) sets in Tq are 
those P T Q that are up- (down-) invariant under ^q; the clopen sets in Tq 
are cycles under G. The inherited topology on X T QTM", and the continuity 
properties, are more complicated, and under current investigation. 

Metric structure on the state space of an LTS model can be used to define 
explicit metric tolerance relations that allow us to express such properties as 
being within e of a set, for a particular e > 0. Again, the resources of modal logic 
come into play. For X a metric space and e > 0, define a relation of e-tolerance 
or e-indiscernability (e) : X X by: 

X (e) x' iff dx(x,x') < e (21) 

Such a relation is reflexive and symmetric, but not transitive. My source for the 
notion of a tolerance relation is Smyth’s [37]. A motivating idea in that paper, 
which is traced back to Poincare’s The Value of Sciencel90fl) and independent- 
ly, to the topologist Zeeman in the early 1960’s, is that perceptual or physical 
continua, as opposed to the idealized continua of classical mathematics, are of 
finite or countable cardinality and are structured by a relation of indiscernability 
that is reflexive and symmetric, but not transitive. In [1] Ch.l, the relation (e) 
goes by the name V). 

Formally, we extend the alphabet X of transition labels with a new symbol e. 
Interpreting the new modalities (e) and [e] in the standard way by the pre-image 
operators cr(e) and r(e), the sentence {e)Lp denotes the e-ball around ||v?||^, or 
the e- closure of ||(y9|] - that is, the set of states within e of some point in || 9 ?j| , 

while \e]y} denotes the e-interior of - that is, the set of states all of whose 
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e-neighbors are in ||</3||^ . The modalities for symmetric and reflexive relations 
are axiomatized by the modal logic KTB; see [9] §4.3. 

The combination of topological and tolerance structure opens up new possi- 
bilities. For example ([1] Ch.l, Corollary 2), if : X X is u.s.c. in a compact 
metric space X, then for each closed set ||(/?|| T X , and each e > 0, there is a 
(i > 0 such that the sentence 



{S){a)ip {a){e)ip 



( 22 ) 



is true in 971. 

Metric tolerance structure can be used to define “imperfect precision” hybrid 
trajectories. In the LTS model of a hybrid automaton Ti, suppose that on 
each projection Xq T M", we have a metric tolerance (5^) : Xq Xq for 
some given 6q > 0. Then instead of considering “perfect precision” trajectories 
formed from the simple alternation of constrained evolution and controlled jump 
relations, as in (3), we might want to consider transition sequences: 

^qo '^qo ”^qo,gi '^qi "^91,92 "^92 j ^ 9 fc_l ”^9fc-i,9fc ^9fc (23) 

Operationally, this can be construed as allowing metric “gaps” of up to size 
6q between the decision to make a controlled switch Cq^qi , and the point at 
which such a switch actually occurs. Defining (6) : X X to be the union of 
each of the lifted relations {Sq), the dynamics of the class of all “5-imperfect” 
hybrid trajectories with finite discrete traces are captured by the dual fixed-point 
modalities 

(h^) 99 = fj,Z. {e)ip \/ {e){5){c)Z and [hs] V? — izZ. [e](^ A [e] [5] [c]Z (24) 

Alternatively, one could “relax” the definition of the constrained evolution rela- 
tion, and take 



{eq)Z ^ {6q)InVq A {{q){Z A InVg) 



that is, Cq = fqC] {InVq T <j{5q)InVq), where the revised convexity property is: 
(fg)Inv, A (f,)(5,)Inv, ^ (55)Inv, 

which says: curves along (j)q that start in InVq and end in cr{5q)lnvq lie inside 
a{5q)lnvq. 



5 Deductive Proof Systems 

We present simple Hilbert-style axiomatic proof systems for the logics of interest. 
The axiomatizations arc not intended to be minimal; rather, they arc meant to 
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serve as a useful reference list. In particular, we give the axioms and rules for 
both of the dual diamond and box modalities. Kozen’s axiomatization [23] 
forms the foundation, with extensions developed in a modular fashion. So far, we 
have identified S4 for topological and relational pre-order modalities, and KTB 
for tolerance relations. A further candidate is S5, the modal logic of equivalence 
relations', we can give modal representation to any partition of the state space 
of our choosing; bisimulation equivalences spring to mind. S5 is also the base of 
logics of knowledge [16]; the knowledge of an agent is modeled by the equivalence 
relation of indistinguishability relative to its knowledge base. 

Equivalent Gentzen sequent-style proof systems for the /u-calculus are pre- 
sented in [5], [8], and also in [40]. 



Definition 9. The Hilbert-style proof system for the logic 1 j^ has the following 
axioms: for transition labels a £ A, propositional variables Z,W ^ PVar, and 
form,ulas p £ S), 

CP : axioms of classical propositional logic 



\J-{a) : 


(a)(Z VVE) {{a)Zy {a)W) 


: (a)ff ^ ff 


A-[a] : 


[a]{Z AW) ^ {[a]Z A[a]W) 


tt-[a] : [a]tt ^ tt 


p-f.p. : 


p[Z pZ.p] — > pZ.p 


n-f.p. : vZ.p p[Z vZ.p] 



and the inference rules, for formulas p.if E ■ 



modus ponens: 
substitution: 

{a) -monotonicity: 



[a] -monotonicity: 








if 






T. 




p[Z 


:= 


r if] 


T 




if 


[a)p 




{a)if 


T 






[a\p 




[a\if 



p-least-f.p.: 



v-greatest-f.p.: 



Hoare composition: 



p[Z 


:=if]^if 


pZ.p if 


if - 


> p[Z := if] 


if 


vZ.p 


if { 


a)x X {b)T 


if 


-> {a){b)x 


if 


[a]x X ^ [b]T 


if 


[a][b]x 



Hoare composition: 
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We write: T ip for formulas p> G S) if there is a proof of p in L^. 

The axioms and monotonicity rules for (a) and [a] together assert they are 
normal diamond (possibility) and box (necessity) modalities ([9] Ch. 4); they 
are equivalent to system K (for Kripke), the logic of generic binary relations. 
In the language of [26], (a) denotes a normal and finitely additive operator on a 
Boolean algebra. The Hoare composition rules follow readily from monotonicity. 
As always, we assume substitutions (f[Z := -i/;] are legitimate ones; i.e. no capture 
of free variables. 

The axioms and rules for the fixed-point quantifiers assert what they ought: 
that jiZ.p {vZ.p) is the least (greatest) fixed point of the operator defined by 

if. 

Each of the rules is readily verified to be truth-preserving^ in the sense that 
for any LTS model 9It, if the hypotheses of a rule is true in 91T then the conclusion 
is true in SOT. From the verification that the each of the axioms is true in every 
LTS model, we then get soundness: if T Lp then dJt \= p for all LTS models 
2)T of signature (^, S). 

Definition 10. The Hilbert-style proof system for the logic + S4 in the lan- 
guage S) is obtained from that of by adding the normality axioms 

and rules for <> and □, together with: for propositional variables Z G PVar, 

TO : Z ^ OZ TD : DZ ^ Z 

40 : OOZ ^ OZ 4D : DZ ^ DDZ 

The proof system for the logic L^, + S4 +Ca is that of + S4 together with 
Ca, where Ca is one or more of the semi- continuity axiom schemes: 

usc(a) : 0{a)Z {a)OZ usc[a] : [a]DZ ^ □[a]Z 

Isc(a) : (a)DZ ^ □(a)Z lsc[a] : 0[a]Z ^ [ajOZ 

In the relational (preorder) semantics for S4, the T axioms correspond to 
reflexivity, while the 4 axioms correspond to transitivity. Extensions of the Hoare 
composition rules: 

ip [a]Dx X [h]Oy , 0 ^ {a)Ox X 

ip [a][6]n(/J ip {a){b)\T\p 

can be derived in the systems + S4 -|- usc[a] -\- usc[b] and X^ -|- S4 -f lsc{a) -|- 
Iscifi) respectively. 

Definition 11. The Hilbert-style proof system for the logic + KTB in the 
language U {e}) is obtained from that of by adding the normality 

axioms and rules for (e) and [e]; the axioms T(e) and T[e]; and also: 

B(e): {e)[e]Z^Z B[e] : Z^[e](e)Z 

The B axioms express that tolerance relations (e) arc symmetric. 
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Definition 12. The Hilbert- style proof system, for the logic + S5 in the lan- 
guage I7U{T}) is obtained from that of by adding the normality axioms 

and rules for (T) and \T]; the axioms T(T), T[T], 4(T) and 4[T]; and also: 

5(T) : {T)[T]Z [T]Z 5[T] : {T)Z [T]{T)Z 

The 5 axioms express that T is a Euclidean relation: \i x T y and x T z 
then y T z. And reflexive, transitive and Euclidean binary relations are exactly 
equivalence relations. Under the knowledge interpretation of S5, the axiom 5[T] 
is usually referred to as the axiom of negative introspection: [T]-i[T](^, 

which reads: “if it is not the case that agent A knows p, then agent A knows 
that it is not the case that she knows (f” . 

Walukiewicz has recently established the completeness of the Kozen axioma- 
tization with respect to the standard set-theoretic semantics for the /i-calculus. 

Theorem 1. ([39], [40]) Soundness and Completeness of (set-theoretic se- 
mantics) 

For all formulas G E), 

T(f iff 9H 1= 99 for all LTS models of signature (<?, S). 

The completeness part of the cited theorem is stated in the form: if g? is 
unsatisfiable in every LTS model 9H, i.e. = 0 for all assignments f in V{X), 

then -i (/5 is provable in L^.Walukiewicz’s proof is very intricate, proceeding by 
first contracting to a subclass of “nice” formulas, and then producing a “tableaux 
refutation” of unsatisfiable formulas of nice form, where such a refutation in 
turn implies that the negation of the given formula is provable in L^. Topics 
of continuing enquiry include whether the Walukiewicz proof can be extended 
to cover specific modal enrichments of L^, and the relationship between his 
tableaux refutation system and a tableaux proof system for the /u-calculus and 
polymodal extensions, in the style of [35] and [10]. 

The algebraic semantics of Kwiatkowska et al. [5] , [8] , provide a framework 
for extending Stone duality theory to the algebra of fixed-points. Their proof 
of completeness for modal /r-frames starts with the Lindenbaum algebra 
of formulas in A) modulo provable equivalence in L^, then realizes the 

abstract / 7 .-algcbra as a canonical LTS model with state space the Stone 
space X — UltiyiFi,^) of (Boolean) ultrafilters in together with the canonical 
/i-algebra — Clop{Ult{!F-L^)) — of subsets of X clopen in the Stone 
topology. For each a G A, and 9Jt = the relations on X are defined by: 

frrt 

X ^>—1 y iff (V^ G Al,j)[ [o ](/3 G X ^ G j/ ]. The formal statement of the result 
is as follows. 

Theorem 2. ([5]) Soundness and Completeness of (algebraic semantics) 

For all formulas g> G A), 

T(f iff (for, A) \= p: for all modal p- frames (971, A) of signature {d>, A). 
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In [8] §6, it is established if (971,^) is a descriptive modal /i-frame, then 
(SOI, A) is in semantic agreement with SOT. In particular, the canonical frame 
) is descriptive, and thus in semantic agreement with the underlying 
LTS model . Thus the “easy” algebraic proof of completeness can be used 
to give an alternative proof of completeness of with respect to the standard 
set-theoretic semantics, as stated in Theorem 1. 

The Kwiatkowska algebraic completeness proof extends quite smoothly to 
normal polymodal extensions of the /x-calculus, including topological S4 exten- 
sions with semi-continuity axioms. For example, if L = + S4 + {usc[a] + 

lsc(a)}aex', the topology on the canonical model 9 IIl comes from a relation 
on X = Ult{tF'L,) defined in the same way as the relations as above. The 
S4 axioms ensure that the relation ^ is a preorder, so the topology is Alexan- 
droff, and from the semi-continuity axiom schemes, one proves that each of the 
relations have the corresponding semi-continuity property. A more detailed 
treatment is given in [12]. 



6 Discussion 

We have developed a family of expressively rich and usable logical systems and 
broadened horizons for the formal analysis of hybrid dynamical systems. In addi- 
tion to those mentioned in the text, further lines of enquiry include the following. 

— Investigation of non-deterministic continuous dynamics, in the form of set- 
valued or parametrized semi-flows, and their topological properties. Our 
relation-based view of dynamics is of course conducive to such generaliza- 
tions. 

— A deeper investigation of relations (definable families) in o-minimal struc- 
tures, and of the use of finite cell-decomposition in the construction of topo- 
logical bisimulations. 

— Further investigation of finite sub-topologies of the standard topology on 
X T K”, and semi-continuity properties of relations in such topologies, pur- 
suing themes developed in [11]. 

— Application to hybrid systems of the theory of knowledge in multi-agent 
settings and its formalization in S5 based logics of knowledge. 

— LTS models and /r-calculus specifications of hybrid petri nets. One approach 
is to take the state space A to be a set of finite partial functions x : P K 
(equivalently, variable- length vectors over M), where P is the hnite set of 
places of the net. 

— Application of game-theoretic methods for the /i-calculus, and related work 
on automata over transition systems; e.g. [25], [22]. 

— Investigation of tableaux proof systems for polymodal logics and the /i- 
calculus, in the style of [35] and [10]. 

— Investigation of Intuitionistic (constructive) logics for hybrid systems, using 
topological semantics and S4 as a bridge between the classical and construc- 
tive worlds. 
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Abstract. First we give a short description of the Extended Linear 
Complementarity Problem (ELCP), which is a mathematical program- 
ming problem. We briefly discuss how this problem can be used in the 
analysis of discrete event systems and continuous variable systems. Next 
we show that the ELCP can also be used to model and to analyze hybrid 
systems. More specifically, we consider a traffic-light-controlled intersec- 
tion, which can be considered as a hybrid system. We construct a model 
that describes the evolution of the queue lengths in the various lanes (as 
continuous variables) as a function of time and we show that this leads to 
an ELCP. Furthermore, it can be shown that some problems in the anal- 
ysis of another class of hybrid systems, the “complementary-slackness 
systems” , also lead to an ELCP. 



1 Introduction 

The main purpose of this paper is to show that the Extended Linear Comple- 
mentarity Problem (ELCP) — which is a kind of mathematical programming 
problem — can be used to model and to analyze certain classes of hybrid sys- 
tems. The formulation of the ELCP arose from our research on discrete event 
systems. Furthermore, the ELCP can also be used to analyze some classes of 
continuous variable systems (i.e., systems that can be modeled using difference 
or differential equations). Since hybrid systems can be considered as a merge of 
discrete event systems and continuous variable systems, this leads to the question 
as to whether the ELCP ean also be used in the analysis of hybrid systems. We 
show that this is indeed the case. More specifically, we consider a traffic-light- 
controlled intersection — which can be considered as a simple hybrid system. We 
show that the evolution of the queue lengths at a traffic-light-controlled intersec- 
tion can be described by an ELCP. Furthermore, the ELCP can also be used to 
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model another class of hybrid systems, the so-called “complementary-slackness 
systems” . 

This paper is organized as follows. In Section 2 we introduce the Extended 
Linear Complementarity Problem. In Section 3 we briefly discuss how the ELCP 
can be used to model and to analyze certain classes of discrete event systems, 
continuous variable systems and hybrid systems. In Section 4 we consider a 
traffic-light-controlled intersection and we show how the evolution of the queue 
lengths in this system can be described by an ELCP. Finally, we present some 
conclusions and directions for future research in Section 5. 

2 The Extended Linear Complementarity Problem 

The Extended Linear Complementarity Problem (ELCP) is an extension of the 
Linear Complementarity Problem, which is one of the fundamental problems in 
mathematical programming [3]. The ELCP is defined as follows: 

Given A € B e c e IR.^, d G IR,® and m subsets (f>i, 02, . . . , 

(j>m. of {1, 2, . . . ,p}, find X € IR" such that 

m 

(Ax (g) c)j = 0 (1) 

j=i ie4>j 

subject to Ax T c and Bx = d, or show that no such x exists. 

Equation (1) represents the complementarity condition of the ELCP. One possi- 
ble interpretation of this condition is the following: since Ax T c, (1) is equivalent 
to 

Vj G {1, 2, . . . , m} : (Ax (gi c)i = 0 . 

i&<tj 

So we could say that each set 4>j corresponds to a group of inequalities of Ax T c 
and that in each group at least one inequality should hold with equality, i.e., the 
corresponding residue should be equal to 0: 

Vj G {1,2,..., m} : G 0j such that (Ax ® c)i — 0 . 

In general, the solution set of the ELCP defined above consists of the union of 
faces of the polyhedron defined by the system of linear equations and inequalities 
(Ax T c and Bx — d) of the ELCP. In [6,7] we have developed an algorithm 
to compute the complete solution set of an ELCP. This algorithm yields a de- 
scription of the solution set of an ELCP by vertices, extreme rays and a basis of 
the linear subspace corresponding to the largest affine subspace of the solution 
set. In that way it provides a geometrical insight in the entire solution set of the 
ELCP and related problems. 

We shall now give a brief description of the ELCP algorithm of [6,7]. The 
algorithm consists of two parts: 
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— First we determine the set of (finite) vertices A’®", the set of extreme rays 

and a basis of the linear subspace corresponding to the largest 

affine subspacc of the solution set of the ELCP. This is done by iteratively 
solving the system Ax T c, Bx = d, whereby in the fcth step (fc = 1, 2, . . . ,p+ 
q) we compute the intersection of the current solution set with the half-space 
or hyperplane determined by the fcth inequality or equality. We also remove 
solutions that do not satisfy the complementarity condition. 

— Next we determine the set A of maximal cross-complementary pairs of sub- 
sets of A’®’"* and A’®". A pair (A®’"*, A®") is cross-complementary if the sum 
of any nonnegative combination of the elements of A®’'* and any convex com- 
bination of the elements of A(®" satisfies the complementarity condition. 
The set A is determined using a kind of backtracking algorithm: we start 
with a pair of the form (0, {x|}) with e A®" and then we keep on adding 
new elements of A’®’'* and A®" to the current pair in a systematic way until 
we obtain a pair that is not cross-complementary any more^. In that case we 
do a backtracking step. This continues until we have obtained all maximal 
cross-complementary pairs. 

Now any solution x of the ELCP can be written as 

^ + Y + Y ( 2 ) 

C.-SA'-n 

for some pair (A®’'*, A®") e A with Afc e R, Kk T 0, p,k T 0 and E — 1- 

k 

For more information on the ELCP algorithm and for a worked example the 
interested reader is referred to [6]. 

In [6,7] we have also shown that the general ELCP with rational data is an 
NP-hard problem. 



3 The ELCP and discrete event systems and continuous 
variable systems 

In this section we briefly discuss how the ELCP can be used in the analysis 
of certain classes of discrete event systems (such as max-linear discrete event 
systems) and of certain classes of continuous variable systems (such as, e.g., 
piecewise-linear resistive electrical circuits). 

3.1 The ELCP and max-linear time-invariant discrete event systems 

Typical examples of discrete event systems (DESs) are flexible manufacturing 
systems, subway traffic networks, parallel processing systems, telecommunication 

^ It can be shown that it is sufficient to test only one combination of the elements of 
and a)’" to determine whether the pair (A^®^*, A)’") is cross-complementary or 

not. 
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networks and logistic systems. The class of the DESs essentially contains man- 
made systems that consist of a hnite number of resources (e.g., machines, commu- 
nications channels, or processors) that arc shared by several users (e.g., product 
types, information packets, or jobs) all of which contribute to the aehievement of 
some common goal (e.g., the assembly of products, the end-to-end transmission 
of a set of information packets, or a parallel computation). 

One of the most characteristic features of a DES is that its dynamics are 
event-driven as opposed to time-driven: the behavior of a DES is governed by 
events rather than by ticks of a clock. An event corresponds to the start or 
the end of an activity. If we consider a production system then possible events 
are: the completion of a part on a machine, a machine breakdown, or a buffer 
becoming empty. 

In general, the description of the behavior of a DES leads to a model that is 
nonlinear in conventional algebra. However, there exists a class of DESs for which 
the model is “linear” when we express it in the max-plus algebra [1,2,4], which 
has maximization and addition as basic operations. DESs that can be described 
by such a “linear” model are called m, ax-linear DESs. Loosely speaking we could 
say that the class of max-linear DESs corresponds to the class of deterministic 
time-invariant DESs in which only synchronization and no concurrency occurs. 

The basic operations of the max-plus algebra are maximization (represent- 
ed by T) and addition (represented by T). There exists a remarkable analogy 
between the basic operations of the max-plus algebra on the one hand, and the 
basic operations of conventional algebra (addition and multiplication) on the 
other hand. As a consequence many concepts and properties of conventional al- 
gebra (such as Cramer’s rule, eigenvectors and eigenvalues, the Cayley-Hamilton 
theorem, . . .) also have a max-plus-algebraic analogue (see, e.g., [1]). Further- 
more, this analogy also allows us to translate many concepts, properties and 
techniques from conventional linear system theory to system theory for max- 
linear time-invariant DESs. However, there are also some major differences that 
prevent a straightforward translation of properties, concepts and algorithms from 
conventional linear algebra and linear system theory to max-plus algebra and 
max-plus-algebraic system theory for DESs. 

If we write down a model for a max-linear DES and if we use the symbols 
T and T to denote maximization and addition^ we obtain a description of the 
following form: 



x{k+l) ^ AT x{k) T BJ u{k) (3) 

y{k) = CTx{k) , (4) 

where x is the state vector, u the input vector and y the output vector. For a 
manufacturing system u[k) would typically represent the time instants at which 

^ For matrices A and B these operations are defined by {A 0 B)ij = atj © bij and 

{A®B)ij = aik^bkj. Note that these definitions closely resemble the definitions 

k 

of matrix sum and matrix product of conventional algebra but with + replaced by 
0 and X replaced by 0. 
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raw material is fed to the system for the {k ® l)st time; x{k) the time instants 
at which the machines start processing the kth batch of intermediate products; 
and y{k) the time instants at which the fcth batch of finished products leaves the 
system. In analogy with the state space model for linear time-invariant discrete- 
time systems, a model of the form (3) — (4) is called a max-linear time-invariant 
state space model. 

r 

Let X, r G M. The rth max-plus-algebraic power of x is denoted by x® and 
corresponds to rx in conventional algebra. 

Now consider the following problem: 

Given p\ + p 2 positive integers mi, . . . , rnp^^p^ and real numbers aki, 

bk and c^ij for k — 1, . . . ,pi + p 2 , i ~ 1, ■ ■ ■ , rnk and j = 1, . . . , n, find 

X G H” such that 

mfc n 

0aHT0x/"'“^ =bk 

i—1 j^l 

rrik n 

0afc,T(g)x,®'''‘^ Tbk 

i=i j=i 

We call (5) - (6) a system of multivariate max-plus-algebraic polynomial equalities 
and inequalities. Note that the exponents may be negative or real. 

In [6,10] we have shown that the problem of solving a system of multivari- 
ate max-plus-algebraic polynomial equalities and inequalities can be recast as 
an ELCP. This enables us to solve many important problems that arise in the 
max-plus algebra and in the system theory for max-linear DESs such as: com- 
puting max-plus-algebraic matrix factorizations, performing max-plus-algebraic 
state space transformations, computing state space realizations of the impulse 
response of a max-linear time-invariant DES, constructing matrices with a given 
max-plus-algebraic characteristic polynomial, computing max-plus-algebraic sin- 
gular value decompositions, computing max-plus-algebraic QR decompositions, 
and so on [6,7,8,9,10]. 

Although the analogues of these problems in conventional algebra and linear 
system theory are easy to solve, the max-plus-algebraic problems are not that 
easy to solve and for almost all of them the ELCP approach is at present the 
only way to solve the problem. 

For more information on the max-plus- algebra and on max-plus-algebraie 
system theory for discrete event systems the interested reader is referred to 
[1,2,4,6,15] and the references given therein. 



for fc = 1, . . . ,pi , (5) 

ioi k = Pi -\- 1, ... ,pi + p 2 ■ (6) 



3.2 The ELCP and piecewise-linear resistive electrical circuits 

In this section we consider electrical circuits that may contain the following el- 
ements: linear resistive elements, piecewise-linear (PWL) resistors (the resistors 
are not required to be either voltage or current controlled), and PWL controlled 
sources (all four types) with one controlling variable (the characteristics may 
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di (A = oo) 





{x)i 



Fig. 1. A oiie-dinieiisioiial PWL curve in characterized by n + 1 breakpoints 
xq, . . . , Xn and two directions do and d\ . The points on this curve can be pa- 
rameterized by (7) where A is a real continuous parameter. 



be multi-valued) . These electrical circuits can be considered as examples of con- 
tinuous variable systems (i.e., systems that can be modeled using difference or 
differential equations) . In this section we shall show that by using an intelligen- 
t parameterization of the PWL characteristics the equations that describe the 
relations between the voltages and currents in these electrical circuits can be 
reformulated as (a special case of) an ELCP. For sake of simplicity we consider 
only two-terminal resistors since they can be described by a one-dimensional 
PWL manifold^. 

If X is a vector, then we define x+ = max(x, 0) and — max(®x, 0), where 
the operations are performed componentwise. An equivalent definition is: 

X = x^ <S> x~ , x^ ,x~ T Q , (x^)^x~ = 0 . 

It is easy to verify that a one-dimensional PWL curve in IR,^ characterized 
by n-|- 1 breakpoints xq, . . . , x„ and two directions do and di (see Figure 1) can 
be parameterized as follows [5,23]: 



X 



n 

xo + doX^ + (xi !g) xo)A+ + ^ (xfc (8> 2xfe_i + Xk-2){X ®k + 1)+ + 

fc =2 



(di (g) x„ + x„_i)(A (g) n)+ , 



( 7 ) 



® If we allow multi-terminal nonlinear resistors, which can be modeled by higher- 
dimensional PWL manifolds, we shall also obtain an ELCP (Sec [5]). 
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with A £ M. Introducing auxiliary variables = A <8) z yields a description of 
the following form: 



X = xo + Ay + By^ 
C{y+ ^Siy^) ^ d 
y+,y^ T 0 
{y+Vy- = 0 



where 



y 

y^ 

A- 

A+ 



B 



[\- K ■■■ K]" 

tA+ A+ ... A+]^ 

[do 0 ... 0] 

[ 0 Xi® Xq X2 ® 2xi + Xo 

'101 0... o' 

1 0 01 ... 0 

1 0 O...01 



c = [ 1 2 ... n]^ 



di®Xn+ Xn-l 



If we extract all nonlinear resistors out of the electrical circuit, the resulting 
A^-port contains only linear resistive elements and independent sources. As a 
consequence, the relation between the branch currents and voltages of this N- 
port is described by a system of linear equations. If wc combine these equations 
with the PWL descriptions (7) of the nonlinear resistors, we finally get a system 
of the form: 

Mw^ + Nw^=q, w^,w^T0, {w^)^{w^)=0 , (8) 

where the vector w contains the parameters A and Aj of the PWL descriptions 
of all the nonlinear resistors. It is easy to verify that (8) can be considered as (a 
special case of) an ELCP. If we solve (8), we get the complete set of operating 
points of the electrical circuit. 

In a similar way we can determine the driving-point characteristic (i.e., the 
relation between the input current and the input voltage) and the transfer char- 
acteristics of the electrical circuit [23]. 

In general, the behavior of an electrical network consisting of linear resistors, 
capacitors, inductors, transformers, gyrators and ideal diodes can be described 
by a model of the form 



x{t) = Ax{t) + Bu{t) 
y{t) — Cx{t) + Du{t) 
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subject to the conditions 

y{t) T 0, u{t) T 0, (y{t))'^u{t) = 0 (9) 

(see, e.g., [19]). In order to compute the stationary points of such an electrical 
circuit, we add the condition x{t) = 0, which leads to a Linear Complementarity 
Problem [19]. If we replace (9) by more general conditions of the form Wi T 0, 
Zi T 0, WiZi = 0, where Wi and Zi are components of u, y or x, then we get (a 
special case of) an ELCP. 

3.3 The ELCP and hybrid systems 

In Sections 3.1 and 3.2 we have shown that the ELCP arises in the analysis 
of certain classes of discrete event systems and continuous variable systems. 
Since hybrid systems arise from the interaction between discrete event systems 
and continuous variable systems, and since they exhibit characteristics of both 
discrete event systems and continuous variable systems, this leads to the question 
as to whether the ELCP can also play a role in the modeling and analysis of 
certain classes of hybrid systems. In the next section we shall show that this 
is indeed the case: we study a traffic-light-controlled intersection, which can be 
considered as a simple hybrid system. The evolution of the queue lengths in this 
hybrid system can be described by an ELCP. 

Furthermore, in [19,21,22] Schumacher and van der Schaft consider another 
class of hybrid systems — the “complementary-slackness systems” — typical 
examples of which are electrical networks with diodes, or mechanical systems 
subject to geometric inequality constraints. They develop a method to deter- 
mine the uniqueness of smooth continuations and to solve the associated mode 
selection problem for complementary-slackness systems. When the underlying 
system is a linear system, then this leads to a Linear Dynamic Complementarity 
Problem which can also be considered as a special case of the ELCP [11]. 

Hence, the ELCP can indeed be used in the analysis of certain classes of 
hybrid systems. 

4 Traffic-light-controlled intersections 

4.1 The set-up and the model of the system 

Consider a single intersection of two two-way streets with controllable traffic 
lights on each corner (see Figure 2). For sake of brevity and simplicity we make 
the following assumptions: 

— the traffic lights can cither be red or green, 

— the average arrival and departure rates of the cars are constant or slowly 
time- varying, 

— the queue lengths are continuous variables. 

These assumptions deserve a few remarks: 
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— Adding an all-red or amber phase leads to a similar, but more complex model 
(see [ 12 ]). 

— If we keep in mind that one of the main purposes of the model that we shall 
derive, is the design of optimal traffic light switching schemes, then assuming 
that the average arrival and departure rates are constant is not a serious 
restriction, provided that we use a moving horizon strategy: we compute the 
optimal traffic light switching scheme for, say, the next 10 cycles, based on a 
prediction of the average arrival and departure rates (using data measured 
during the previous cycles) and we apply this scheme during the first of the 
10 cycles, meanwhile we update our estimates of the arrival and departure 
rates and compute a new optimal scheme for the next 10 cycles, and so on. 

— Designing optimal traffic light switching schemes is only useful if the arrival 
and departure rates of vehicles at the intersection are high. In that case, 
approximating the queue lengths by continuous variables only introduces 
small errors. Furthermore, in practice there is also some uncertainty and 
variation in time of the arrival and departure rates, which makes that in 
general computing the exact optimal traffic light switching scheme is utopian. 
Moreover, in practice we are more interested in quickly obtaining a good 
approximation of the optimal traffic light switching scheme than in spending 
a large amount of time to obtain the exact optimal switching scheme. 

Let us now continue with the description of the set-up of the system. There are 
four lanes Li, L 2 , L3 and L4, and on each corner of the intersection there are 
traffic lights (7i, T 2 , T 3 and T4). The average arrival rate of cars in lane Li is 
Aj. When the traffic light is green, the average departure rate in lane Lj is /Xj. 
Let to, tiy t 2 , ts, . . . be the time instants at which the traffic lights switch from 
green to red or vice versa. The traffic light switching scheme is shown in Table 1. 
Define dk = tfe+i ® ffe- Let li{t) be the queue length (i.e., the number of cars 
waiting) in lane Li at time instant t. 

Let us now write down the equations that describe the relation between the 
switching time instants and the queue lengths as continuous variables. 

Consider lane L\. When the traffic light Ti is red, there are arrivals at lane 
Li and no departures. As a consequence, we have 



dh{t) 

dt 



= Ai 



( 10 ) 



Period 


Tl 


T 2 


T 3 


Ti 


to-tl 
tl -t2 
t2-t3 


red 

green 

red 


green 

red 

green 


red 

green 

red 


green 

red 

green 



Table 1. The traffic light switching scheme. 
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Fig. 2. A traffic-light-controlled intersection of two two-way streets. 



for t e {t2k, ^2fc+i) with /c € IN, and 

^l(^2fc+l) = h{t2k) + Ai52fe 



for fc = 0, 1, 2, . . . 

When the traffic light T\ is green, there arc arrivals and departures at lane Li. 
Since then the net queue growth rate is Ai 0 and since the queue length li{t) 
cannot be negative, we have 

dli(f) _ JAi 0Mi if > 0 
dt ^0 if h{t) = 0 

for t e {t 2 k+i , hk+ 2 ) with fc e IN. So 

^i(f 2 fc+ 2 ) = max(/i(f 2 fe+i) + (Ai 0 Mi)<^2fc+i,0) 

for fc = 0, 1, 2, . . . 

Note that we also have 

^i(^2fe+i) = max(li(t2fe) + Ai52fe,0) 

for fc = 0, 1, 2, . . . since h{t) T 0 for all t. 

We can write down similar equations for hitk), h{tk) and h{tk)- 
So if we define 



h{tk) 




Ai 




Ai (8) /Xi 


h{tk) 


, bi 


A2 0 jJi2 


) h — 


A2 


h{tk) 


A3 


A3 fls 


}i{tk) _ 




A4 0 /i4 




A4 
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then we have 



X2k+i = max(x2fc + hi52k, 0) (12) 

X 2 k +2 = max(x2fc+i + 62^2fc+i, 0) (13) 



for fc = 0, 1, 2 , . . . 



Remarks: 

— The traffic-light-controlled intersection can be considered as a hybrid system 
that has the time and the queue lengths as state variables, and that can 
operate in two regimes characterized by differential equations of the form 
(10) or (11) depending on the value of a discrete control variable that can 
have the value “red” or “green” . 

— The model we have derived is different from the models used by most other 
researchers due to the fact that we consider red-green cycle lengths that may 
vary from cycle to cycle. Furthermore, we consider non-saturated intersec- 
tions, i.e., we allow queue lengths to become 0 during the green cycle. Some 
authors (see, e.g., [16,20]) only consider models for oversaturated intersec- 
tions, i.e., they do not allow queue lengths to become equal to 0 during the 
green cycle. In that case the maximum operator that appears in (12) - (13) is 
not necessary any more, which leads to a simpler description of the behavior 
of the system. However, in [13] we have shown that, when we want to design 
optimal traffic light switching schemes, applying a model for oversaturated 
intersections to a non-saturated intersection in general does not lead to an 
optimal traffic light switching scheme. 

4.2 Link with the ELCP 

Let us now show that the system (12) -(13) can be reformulated as an ELCP. 

First consider (12) for an arbitrary index k. This equation can be rewritten as 

follows: 



X2k+1 ^ ^2k + b\S2k 
X2k+1 0 

{X 2 k+i)t = {X 2 k + h52k)i or (X2fc+i)j = 0 for i = 1, 2, 3, 4 , 
or equivalently 

a:2fc+i ® X2k <8> bi52k T 0 
X2k+1 0 

(a;2fc+i <8> X 2 k ® biS 2 k)i {x 2 k+i)i = 0 for all i . 

Since a sum of nonnegative numbers is equal to 0 if and only if all the numbers 
are equal to 0, this system of equations is equivalent to: 



X2k+l ^ X2k ® bi 62 k T 0 
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X2fc+1 T 0 

4 

^(a^2/c+l 0 X2k 0 hi 62 k)i {X2k+l)i = 0 



i=l 



We can repeat this reasoning for (13) and for each index k. 
If we consider N switching time instants and if we define 





Xl 




do 




X2 






X — 


Xn 


and S* = 


1 

1 

1 



we finally get a description of the form 

Ax* + B6* +cT0 
X* TO 

(Ax* + B5* + cYx* = 0 . 



(14) 

(15) 

(16) 



It is easy to verify that the system (14) - (16) is a special case of an ELCP. 

Now we can compute traffic light switching schemes that minimize objective 
functions such as 

— (weighted) average queue length over all queues: 

rtN 






li(t) dt 



I to 



i=l 



0 to 

— (weighted) worst case queue length: 

J 2 = , 

— (weighted) average waiting time over all queues: 

ftN 

li(t) dt 



■h = ^ Wj 



'to 






\i(tM ® to) 



and so on, where Wi > 0 for all i. Furthermore, we can impose extra conditions 
such as minimum and maximum durations for the green and the red time'^, 
maximum queue lengths^, and so on. This leads to the following problem: 



minimize J (17) 

A green time that is too short is wasteful. If the red time is too long, drivers tend 
to believe that the signals have broken down. 

® This could correspond to an upper bound on the available storage space due to the 
distance to the preceding junction or to the layout of the intersection. 
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subject to 



‘^min,r G (j2fc e 5 max,r 
^min.g G <^2fc+l G S max.g 
Xfc G 3!^max 
X2k+i = max(x2fc + &i^2fc, 0) 
a;2fc+2 = niax(x2fc+i + 62(52fc+i,0) 



for 


k 


G 


q{N) 


(18) 


for 


k 


G 


f3{N) 


(19) 


for 


k 


= 


1,2,. ..,N 


(20) 


for 


k 


G 


a{N) 


(21) 


for 


k 


G 


m) , 


(22) 



with 



a{N) 




iV ® 1 



and f3{N) 





where [xj is the largest integer that is less than or equal to x. 

It can be shown [13] that the objective function J2 (i.c., the (weighted) worst 
case queue length) is convex as a function of the d^’s, which implies that problem 
(17) - (22) with J — J 2 can be solved efficiently (if there is no upper bound on the 
queue lengths, or if we deal with constraint (20) by introducing a convex penalty 
term if some components of .Xmax are finite). However, the objective functions 
Ji and J3 are neither convex nor concave. Our computational experiments have 
shown that in order to solve problem (17) -(22) with J — J\ 01 J — Js using 
constrained optimization (with, e.g., sequential quadratic programming) several 
initial starting points are necessary to obtain the global minimum. 

Using the procedure given above the system (18) -(22) can be rewritten as 
a system of the form 



Ax* + BS* + ce 0 


(23) 


X* e 0 


(24) 


Bx* + m* + f E 0 


(25) 


X* + BS* + c)'^ X* = 0 , 


(26) 



which is again a special case of an EL CP. In order to determine the optimal traffic 
light switching scheme we could hrst determine the solution set of the ELCP and 
then minimize the objective function J over this solution set. Our computational 
experiments have shown that the determination of the minimum value of the 
objective functions Ji and J3 is a well-behaved problem in the sense that using 
a local mininiization routine (that uses, e.g., sequential quadratic programming) 
starting from different initial points always yields the same numerical result 
(within a certain tolerance). Furthermore, it can be shown [13] that J 2 is a 
convex function of the parameters Afc, and that characterize the solution 
set of the ELCP (cf. (2)). 

The algorithm of [6,7] to compute the solution set of a general ELCP requires 
exponential execution times. This implies that the approach sketched above is 
not feasible if the number of switching cycles N is large. However, in [12] we 
have developed efficient methods to determine suboptimal traffic light switching 
schemes for the model (23) -(26): for the objective functions Ji (i.c., (weighted) 
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average queue length) and J 3 (i.e., (weighted) average waiting time) we can make 
some approximations that transform the problem into an optimization problem 
over a convex feasible set, or even into a linear programming problem. This 
approach is computationally very efficient and yields suboptimal solutions that 
approximate the global optimal solution very well. 

For more information on other models that describe the evolution of the 
queue lengths at a traffic-light-controlled intersection and on optimal traffic light 
control the interested reader is referred to [14,17,18,20] and the references given 
therein. 

5 Conclusions and further research 

We have introduced the Extended Linear Complementarity Problem (ELCP) 
and indicated how it can be used in the modeling and analysis of certain classes 
of discrete event systems, continuous variable systems and hybrid systems. More 
specifically, we have shown that for a traffic-light-controlled intersection the evo- 
lution of the queue lengths at the switching time instants can be described by 
an ELCP. 

Topics for further research include: development of efficient algorithms for 
the special cases of the ELCP that appear in the analysis of specific classes of 
hybrid systems, investigation of the use of the ELCP to model and to analyze 
other classes of hybrid systems, and extension of our model for a traffic-light- 
controlled intersection to networks of intersections. 
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Abstract. This paper presents the results of simulation and control 
experiments using a recently proposed method for real-time switching 
among a pool of controllers. The switching strategy selects the current 
controller based on neural network estimates of the future system perfor- 
mance for each controller. This neural-network-based switching controller 
has been implemented for a simulated inverted pendulum and a level con- 
trol system for an underwater vehicle in our laboratory. The objectives 
of the experiments presented here are to demonstrate the feasibility of 
this approach to switching control for real systems and to identify tech- 
niques to deal with practical issues that arise in the training of the neural 
networks and the real-time switching behavior of the system. This ex- 
perimental work complements on-going theoretical investigations of the 
method which will be reported elsewhere. 



1 Introduction 

Recently there has been an interest in switching control schemes for adaptive 
control [1,2, 8, 9], and fault-tolerant control [10]. A switching control scheme based 
on neural network estimates of performance indices for the candidate controllers 
was proposed in [5]. The objectives of theoretical investigations of switching con- 
trollers are to establish sufficient conditions for closed-loop stability and bounds 
on closed-loop performance. This paper reports on empirical investigations of 
the switching control scheme proposed in [5] and considers some of the prac- 
tical issues that arise in the implementation of switching control schemes. The 
basic rule of the switching strategy is to select the controller at each sampling 
time with the lowest performance index, as estimated by the ensemble of neural 
networks. This strategy is motivated by the so-called min-switch rule for multi- 
ple Lyapunov functions [4,7]. Each of the performance indices being estimated 
is a cost-to-go function which, under mild assumptions, would be a Lyapunov 
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function for the system under control of the corresponding state-feedback law. 
If these Lyapunov functions were known precisely, the min-switching strategy 
would lead to an asymptotically stable system. The strategy investigated in this 
paper addresses the practical issue of how to implement the min-switching strat- 
egy for real systems. For real systems, the true dynamics and Lyapunov functions 
are never known exactly. We demonstrate empirically that neural networks can 
be used to estimate cost-to-go functions effectively, and that acceptable closed- 
loop asymptotically stable behavior can be achieved using these estimates. This 
paper focuses on performance-based switching. The complete switching strategy 
includes an estimate of the region of stability for each controller; a controller is 
not available if the current state is outside the region of stability for the closed- 
loop system under that controller. Results related to the estimation of regions of 
stability using neural networks are presented in [6] . Typically this min-switching 
rule avoids selecting controllers for which the state of the system is outside the 
stability region, however. Since the performance index is very large (possibly 
infinite) for the controllers for which the state is outside the stability region, we 
found the stability region estimates to be unnecessary in the experiments report- 
ed in this paper. The paper is organized as follows. The next section presents a 
real-time infrastructure for fault-tolerant control as a primary motivation for the 
switching control problem considered in this paper. Two applications are also in- 
troduced for which multiple controllers have been designed and the problem is to 
establish switching rules to select the appropriate controllers in real time. Section 
3 presents the neural-network-based switching strategy. Section 4 describes the 
method for training the neural networks to estimate closed-loop performance 
based on the concept of neural dynamic programming. Data is presented for 
training the neural networks for the two example applications. We then consider 
the effectiveness of the switching control strategy in section 5 where experimental 
results arc presented for the two experimental examples. It is observed that chat- 
tering can occur due to the errors in the neural network estimates and methods 
for eliminating the chattering are discussed. The concluding section summarizes 
our experience with the neural-network-based switching control and identifies 
several directions for further investigations, both experimental and theoretical. 

2 Applications of Switching Control Strategies 

Switching control strategies are generally considered for situations where there 
are multiple operating regimes for a system and different controllers are designed 
for each operating regime. Switching control has also been proposed as an ap- 
proach to adaptive control to deal with unknown plants. In this paper we consider 
a version of the former scenario where different controllers are available to sta- 
bilize a given system, but with different performance objectives. One controller 
might be designed to provide good transient performance for small deviations 
from the equilibrium, whereas another controller may be designed to drive the 
system quickly back to the equilibrium when large deviations are detected. The 
latter controller may perform poorly when the system is near the equilibrium, 
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so it is desirable to switch back to the first controller at some point. In this sec- 
tion we first describe a real-time infrastructure that relies on switching control 
strategics to provide fault tolerance against errors in untested control code. This 
architecture provides a practical motivation for the switching control strategy 
proposed in [5] and is the platform used for the control experiments presented 
in this paper. We then present two applications for which multiple controllers 
are available as test cases for the empirical investigations. 



2.1 SIMPLEX Architecture 

One of the main motivations for developing the controller switching strategy 
presented in this paper is to provide a method for implementing the switch- 
ing rules in the SIMPLEX architecture, a technology developed at the Software 
Engineering Institute at Carnegie Mellon University to support safe, reliable 
on-line upgrades and modifications to real-time control systems [10]. Figure 1 
shows a basic configuration for SIMPLEX. It consists of three controllers: a safety 
controller, a reliable baseline controller, and the experimental controller repre- 
senting a new untested control module. The basic idea of the SIMPLEX system is 
to guarantee that the baseline controller performance is maintained if there are 
problems with the experimental controller. This is accomplished by monitoring 
the control outputs and system performance when the experimental controller 
is operating on the system, and switching control back to the baseline controller 
if problems are detected. The safety controller is invoked when it is necessary 
to take more extreme action to return the system to the operating region for 
the baseline controller. The controllers run concurrently as separate tasks in a 
real-time multitasking operating system, or even on different machines. SIM- 
PLEX is in charge of all the communication between the parts of the system 
and monitors the performance of the controllers acting on the plant to deter- 
mine which controller should actually be controlling the plant at each instant. 
The SIMPLEX architecture has been applied successfully to the control of sev- 




Fig. 1. SIMPLEX architecture. 
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eral physical systems including single and multiple inverted pendulum systems, 
a plasma deposition system, and a level-control system for a submerged vessel. 
We present two of these systems in the following subsections. /,From a control 
perspective, SIMPLEX corresponds precisely to the multi-controller scenario con- 
sidered in this paper. The controllers are designed for a given plant with similar 
control objectives, in most cases to stabilize the system, but with different re- 
gions of stability and different performance characteristics. Clearly the ability 
for the SIMPLEX system to provide the desired protection against errors in the 
experimental controller depends entirely on the rules used to switch to the safety 
and baseline controllers. These rules are very difficult to determine analytical- 
ly, even for low-dimensional systems. In the current applications of SIMPLEX, 
physical insight and extensive experimentation have been used to create ad hoc 
rules for switching to the safety and baseline controllers. The neural network ap- 
proach proposed in this paper provides a means for determining automatically, 
and with some theoretical motivation, when to switch to the safety controller 
and the baseline controller. The on-line learning capabilities of the neural net- 
works also suggest the possibility of acquiring knowledge of the performance of 
an experimental controller, leading to the eventual transition of the experimen- 
tal controller to the new baseline controller once it has been completely tested 
and verihed. 



2.2 An Inverted Pendulnm 



As one example of an application of controller scheduling we use the well known 
inverted pendulum (IP) with a set of three controllers. One reason for using this 
application is that a physical IP has served as a standard laboratory example for 
the SIMPLEX architecture. The complete IP model and control design is described 
in [5]. We include here a brief summary. The IP model equations used are: 



1 •• 1-2 

Jtx + — ml COS0 6 + BrX O - ml sinO 9 = F 

— m cos9 X -f —ml9 ® - mg sinO = 0 



with the following constraints: 



(^1 F maxi \x\ Xmaxi \x\ tr Vmax 

where 9 is the angle of the pendulum, x is the position of the “cart” (the pendu- 
lum base), m is the effective mass at the end of the pendulum, I is the pendulum 
length, Ji is the inertia of the cart, is the coefficient of friction for the cart, 
and g is gravitational acceleration. The parameters for the physical system in 
the laboratory are: 
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The controllers designed for the system are: a standard linear quadratic regu- 
lator (LQR) controller for the linearized model of the IP around the origin in 
the full state space, a velocity feedback (VF) controller that takes the pendulum 
angle 9 and the velocity of the cart to zero (but ignores the cart position), and 
a sliding mode (SM) controller designed to bring 6 to zero as fast as possible 
using a nonlinear switching rule and saturating control (but without regard to 
the cart position or the cart speed). The need for a switching strategy is clear. 
There are different controllers with different characteristics available for a phys- 
ical system. Given the designs of these three controllers, one would expect the 
performance of the SM controller to be best for larger pendulum angles, the 
VF controller to work well when the position of the cart is large, and the LQR 
controller to provide the best performance for small deviations from the origin 
of the four-dimensional state space. Therefore, there is a need for a supervi- 
sor to schedule those controllers in the best way possible, where ’’best” means 
asymptotic stability with good transient performance. 



2.3 Submerged Vessel 

This application is an experimental system in our laboratory consisting of a 
water tank in which a vessel (” diver” ) can move vertically by changing the size 
of the air bubble inside it. Air is moved in and out of the vessel through a 
flexible tube connected to a cylinder-piston mechanism. Figure 2 illustrates the 
system components and Figure 3 lists the physical parameters for the system. 
The position of the vessel and the size of the air bubble are measured directly 
using ultrasound sensors. A stepper motor controls the piston movement. The 
control objective is to stabilize the vessel at an arbitrary level without allowing 
the vessel to touch the bottom of the tank or emerge above the surface of the 
water. Any equilibrium position for the vessel is open-loop unstable since the 
air bubble compresses (expands) as the vessel goes down(up). There is also a 
significant time delay in the response of the vessel level to changes in the cylinder 
position due to the volume of the tube connecting the piston with the vessel. 
The following model has been developed for the system using first principles, 

y = <S>a\y\y bh (g) fr{y) 

h = <ga{y, h) ® P{y, h)u{t g r) 



with 
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fiy + /2 sgn{y) e 
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where y is the position of the vessel with respect to the water level and h is 
the height of the bubble. For the first equation governing y, the position of the 
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vessel, the first term of the right hand side is the water resistance, the second 
term reflects Archimedes’ principle, and the last term is the Newtonian static 
and dynamic friction. The controlled input is the velocity of the piston that 
pushes the air between the vessel and the cylinder and r is the time delay. 
State constraints are imposed by the bottom of the tank and the water level. 
The control input is limited by the maximum speed of the stepper motor. For 




Fig. 2. Sketch of the compo- Fig. 3. Physical parameters for 
nents of the submerged vessel the submerged vessel system, 
system. 



this example two linear state feedback controllers are available to regulate the 
position of the vessel. One of them is a bang-bang controller (BB) designed 
to operate when the vessel position is far from the desired position. The other 
controller is a linear state- feedback controller (LSF) that operates best when 
the vessel is close to the setpoint. The first controller drives the vessel to the 
setpoint much more quickly than the second controller, but the vessel oscillates 
around the setpoint in a limit cycle if the first controller is applied indefinitely. 
The scheduler must select which controller to apply at each instant to drive the 
vessel to the setpoint with the best performance, which includes some measure 
of the speed at which the vessel reaches the equilibrium and the quality of the 
regulation once the vessel is near the setpoint. 
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3 Neural Network Based Switching Strategy 

As a general formulation of the problem being considered, let a state-constrained 
and control-constrained nonlinear system in its state space formulation be given 

by 



Xfc+l = /(Xfc,Ufe) (1) 

Xfc e S', Uk e D 

where x^ e i?" is the state vector and Ufe e the control input vector. The 
nonlinear function / is assumed to be smooth. The discrete-time state equations 
reflect the sampled-data nature of the computer control system. We assume that 
the state Xfe is observable. The control signal Ufe to be applied on the system 
can be generated by M different state feedback controllers of the form: 

Ufc=ffi(xfc), i = l,2,...,M. (2) 

The goal of the controllers is to take the system to the origin. We are inter- 
ested in developing a strategy to select which controller should be applied to 
the system at each sampling instant in order to achieve stable response with 
good transient performance. In a similar setting there have been some result- 
s reported concerning the stability of a switching controller. Branicky [4] has 
established conditions for a switched autonomous system to be stable in the 
sense of Lyapunov using the concept of Lyapunov-like functions. Malmborg [7] 
examines asymptotic stability and explained chattering as a possible behavior 
generated intrinsically by the switching and not just as an implementation prob- 
lem. To define our scheduling rule we need to define a performance index for each 
autonomous system generated by the application of a specific controller. Let us 
denote the trajectory of the system (1) with the initial state x„ at time step 
k by the application of the state feedback control as x^(xq). We consider 
performance indices for the controllers of the form 

OO 

4(x„) =B* + ^5'=C/*(xUxo)), i = l,...,M (3) 

fc =0 

where 0 < 5 < 1 is a discount factor and U{.) represents the cost function of 
being in a particular state. The proposed approach for selecting the controller 
at each sampling instant is illustrated in Figure 4. Neural networks are used to 
estimate the performance indices JJ at the current state, denoted by J|. The 
index of the control input to be applied for the next period, denoted ik, is then 
selected as 



= arg min {J|(xfc)} 

iG / (x/c ) 



( 4 ) 



where 



7(xfe, Lfc) = {i\i = Zfc_i or i / i](xfc) < LI} 



( 5 ) 
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with Lfe = {Ll, . . . , L^}, and for i = 1, . . . , M 

Expressed in words the switching rule will select the controller with the best 
performance but from a restricted set of controllers called /(xfc,Lfc) consisting 
of the current controller and all the other controllers which have a better perfor- 
mance estimate than the last time they were used. This selection rule is related 
to the conditions stated by Branicky [4] to have stability in the sense of Lya- 
punov. Figure 4 illustrates how the scheduling rule works. When the estimate 
of the performance index of another controller becomes the lowest one and it is 
also lower than the last time that controller was applied - represented by the 
variable - the switching rule selects that controller. An augmented version 
of this rule may include the stability region for each controller as an additional 
requirement for it to be in the set I(xfc, Lfc). It has been pointed out above that 
this was not necessary for the applications in this paper. 



if ^ ^ ffc 
if z — 



( 6 ) 




Fig. 4. Controller switching scheme used and example to show how 
it works. 



4 Neural Network Training 

This section describes a neural network architecture and training techniques to 
estimate the index of performance for a given controller. The proposed approach 
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is based on a modification of a dynamic programming procedure called neuro- 
dynamic programming (NDP) [3]. In NDP, a neural network is used to estimate 
the cost-to-go function. One motivation for using a neural network is for data 
compression: a neural network is much smaller than, say, a table representation 
of the cost-to-go function. The other motivation for using neural networks is the 
availability of techniques to train them to approximate the cost-to-go functions 
using data from the system. To estimate the performance indices J|(x) defined 
in section 3 we chose neural networks of the form: 

i|(x) = b„ + IT„x + IPs <i>{W 2 <^(lTix + bi) + b 2 ), (7) 

where kPj,bj are the weight matrices and threshold vectors, respectively. The 
nonlinear function (/>(.) for the hidden units of the neural network is chosen 
to be the hyperbolic tangent function. The output units are selected as linear. 
Figure 5 shows a diagram of the network architecture. 




Fig. 5. Multilayer neural network. 




Fig. 6. HDP learning procedure diagram 
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Neuro-dynamic programming requires a type of learning procedure that is 
different from the well known supervised learning algorithms. We use Heuristic 
Dynamic Programming (HDP) algorithm [13]. If we denote by Ji{x) the network 
estimate of the future performance for the ith controller, the learning equation 
takes the form 

Jr{xk)^U{xk) + SJl{xk+i) (8) 

where J|*(xfc) is the desired value for the network for state x^. The pointwise 
cost function selected is a standard quadratic form: 

H(x) = x^ P X, P > 0 

Using this cost function for each trajectory, the input-output patterns for the 
network can be computed. An error function for the network can be evaluated 
and a minimization procedure applied afterwards to adapt the parameters of 
the neural network. Figure 6 represents the learning procedure schematically. 
With this procedure we characterize each autonomous nonlinear system result- 
ing from the application of a single controller. In this sense it is different from 
the Q-learning approach [12] that points to the design of a new controller by 
generating a performance index surface resulting from the collective action of 
all the controllers available at each state. In the case of the inverted pendu- 
lum example, three neural networks were trained to approximate the cost-to-go 
function of each closed-loop system generated by closing the loop with the three 
controllers. Using the simulation model, training was performed off-line in each 
case using 5000 random trajectories starting at states uniformly distributed over 
the allowable set of states. A set of 50 trajectories was used as a validation set. 
On-line learning is continued during the simulated control experiments using 
HDP. Equation (8) with 6 = 0.5, is used to generate the input-output patterns 
for the network. A square error function is computed and conjugate gradient 
optimization methods were used to minimize it. Off-line training was stopped 
when the error was 0.02 to prevent overtraining. Figures 7 and 8 show the esti- 
mation achieved for two controllers in a slice of the state space. The shapes are 
expected, close to a quadratic function near the origin and reaching large values 
outside its neighborhood. There is also some symmetry in the surface that can 
be explained from the model and cost functions symmetry. For the submerged 
vessel system two multilayer networks were set up to learn the performance in- 
dex for the two controllers. In this case, 20 experiments were run on the physical 
system for each controller to get the data for training the networks. It should be 
pointed out that the experiments run on the vessel were selected with starting 
points along the region in which we were expecting the system to operate. Five 
additional experimental runs were saved for network validation purposes. Af- 
ter that, the procedure is similar to the previous example. Overtraining criteria 
stopped training when the overall error was about 0.035. Figures 9 and 10 show 
the estimation achieve for the two controllers designed for this example in a slice 
of the state space. Even though the error is bigger than in the previous example 
the shapes of the cost-to-go estimates have similar characteristics to the ones 
found for the pendulum example. 
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Fig. 9. Performance estimate for Fig. 10. Performance estimate for 
controller LSF. ( y = 0 ) controller BB. ( y = 0 ) 



5 Real-Time Control Results 

Using the IP model described in section 2.2 we tested the scheduling policy 
defined by equations (4), (5) and (6) with simulation. Figures 11 and 12 show the 
position of the pendulum and the evolution of the performance index estimates 
during a typical run. We see in Figure 12 that the evolutions of the cost-to-go 
estimates are not monotonically decreasing. This is due to the fact that these 
are estimates of the true cost-to-go functions. Moreover, the discounted cost is 
not guaranteed to be a Lyapunov function for the system. In Figure 11, a slight 
chatter in the response of the switching mechanism can be observed. This is 
commonly observed in switching systems. It is caused by the discontinuity in 
the control signal when the system goes through the switching boundary and 
the new control value returns the system to the switching surface. For a strategy 
like the one we are using in this work, but in which the performance index is a 
Lyapunov function for each controller in closed loop with the system, it has been 
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Performance estimates 




Fig. 11. Switching experiment Fig. 12. Performance index for each 

controller during run. 



shown [7] that in certain cases a chattering control drives the system along a 
surface in the state space where two or more Lyapunov functions are equal. This 
is commonly referred to as a sliding mode. When estimates are used, chattering 
can also be caused by the nonmonotonicity of the performance indices. There 
are several ways in which chattering has been eliminated or at least moderated 
before. The introduction of a boundary layer with many possible variations is a 
common approach [11]. Another way is the introduction of a non- zero minimum 
time between switches to reduce high frequency switching dynamics [2] . Another 
interesting idea is to compute the control that gives you the direction of the 
sliding surface as a way to eliminate the non-smooth behavior. The first two 
approaches can be applied in our method while the latter would not always be 
possible due to the usual lack of a system model. However, in this experiment 
the switching rule takes care of a big part of the possible chatter not allowing a 
controller to be used until its performance index has returned below its lowest 
previous value. For the underwater vessel control system, experimental runs with 
the physical system where performed. Figure 13 shows a switching experiment 
for a step change in the setpoint value for yst of 12 inches. Figure 5 shows 
the estimated performance indices during the run. Chatter is not observed in 
this example. ^Fi'om the Figure 5 we observe that the bang-bang controller 
is preferred for larger values of y. After approximately 5.5 seconds becomes 
smaller than and the scheduler switches to the linear state-feedback controller. 
In Figure 13 we see also an offset in the hnal position of the vessel due to the high 
static friction in the vertical bar along which the vessel slides. Other controllers 
arc being designed to take care of that problem and can be introduced in the 
scheduling policy directly. 
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Fig. 13. Diver position and bubble size of the vessel during a switeli- 
ing experiment. 



Performance Indices 




Fig. 14. Performance indices estimates for controllers for the sub- 
merged vessel during a switching experiment. 

JI - LSF controller. j| - BB controller. 

6 Summary and Discussions 

A technique is presented to perform controller scheduling using a multilayer 
feedforward neural network and neuro-dynamic programming to estimate a per- 
formance index for each controller. Results from two applications are presented 
demonstrating the feasibility of the approach. Current research is focusing on 
learning algorithms to reduce the effect of the estimation error on the closed loop 
behavior. We are also investigating the use of better features as inputs to the 
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network and different network architectures to speed up learning. Furthermore, 
looking at the overall picture, comparisons are being made with other switching 
schemes like model-based Lyapunov function scheduling techniques in terms of 
stability, smoothness and robustness range. Finally, there arc several theoretical 
issues currently under investigation. A recent result by Malmborg et al. [7] on the 
stability of the so-called min- switching rule based on Lyapunov stability theory 
suggests a way to analyze this architecture. Preliminary work in this area has 
been done but it remains under investigation to guarantee the convergence for 
the neuro-estimation learning algorithms and stability of the closed-loop system 
with adaptive controller scheduling. 
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Abstract. In this paper, we present a rigorous design of a Fault Diag- 
nosis and Isolation algorithm. The system is modelled as a hybrid sys- 
tem with a network of parallel components. The requirement is specified 
in Duration Calculus, a dense time temporal logic. We use traditional 
program logic, suitably extended, to verify the discrete component and 
subsequently derive a number of properties of the system. Finally, the 
requirement is shown to be satisfied by proving that it can be deduced 
from the system properties. 



1 Introduction 

In virtually all control systems, some physical variables must be measured. In the 
real world, the sensors which perform this job may be faulty, either temporarily 
or permanently. Therefore, to achieve a high degree reliability, several copies of 
sensors are used to sample the same variable. By comparing the sampled values 
from the sensors, and possibly some analytic reasoning based on the dynamical 
behaviours of the plant, a monitor tries to identify a sensor which is not faulty 
and uses its output as the final sampled value. 

Usually, a Fault Diagnosis and Isolation (FDI) system contains an odd num- 
ber of sensors, and the output is chosen to be one agreed upon by the majority. 
However, there are cases that an even number sensors (which cannot produce 
a majority when exactly half of the sensors are faulty) have to be used. For 
example, it could be that the sensor is very costly, or there is not enough space 
to install yet another sensor. In this paper, we study an FDI system with four 
sensors. As illustrated by the following diagram, the variable to be sampled is 
y, and output is decided by the monitor based on four inputs y\ to 1/4 from 
the sensors. 



* On leave from Department of Automatic Control, Beijing University of Aeronautics 
and Astronautics, Beijing, 100083, P.R. China. Email: gjp@ns.dept3.buaa.edu.cn 

P. Antsaklis et al. (Eds.): Hybrid Systems V, LNCS 1567, pp. 100—121, 1999. 

© Springer- Verlag Berlin Heidelberg 1999 




Rigorous Design of a Fault Diagnosis and Isolation Algorithm 101 




Figure 1: An FDI system. 



Typically, an FDI system should satisfy the following two properties: 1, the 
accumulated time that the final output is not correct should be less than a 
certain fraction of the total elapsed time; 2, permanent faults should be iden- 
tified with related sensors isolated. To offer a high level assurance, we need to 
reason rigorously that the design indeed satisfies the requirements besides the 
traditional validation methods of simulation and testing. 

An FDI can be considered as a hybrid system with interacting continuous 
and discrete components. The output of a sensor is the summation of the nom- 
inal output, the noise function and the fault function. The nominal output is 
continuous obeying certain dynamical laws, the noise function is relatively small 
and the fault function is discrete reflecting instantaneous occurrence of faults. 

In the literature, hybrid systems are often represented as Automata or as 
Phase Transition Systems [8,7]. In our approach, we work with a notation for 
hybrid systems which is closer to how hybrid systems arc programmed. In par- 
ticular, complex digital control algorithms can be directly incorporated in our 
framework as subprograms. Therefore, we call the notation Hybrid Programs. 
As an effective way to verify a hybrid program, it is suggested in [13] to use a 
program logic to verify the discrete component, and incorporate the results into 
a real-time temporal logic which is subsequently used to deduce properties of 
the whole system. This approach has been further studied in [12,11]. 

This paper is organised as follows. In section 2, we briefly review the Duration 
Calculus (DC) [5,6], which is a dense time temporal logic based on the Interval 
Temporal Logic (ITL) [10]. We next introduce the notation of Hybrid Program, 
and the verification method for it. The requirement of the FDI system is analysed 
in section 4. The system is modelled as a hybrid program in section 5 and verified 
in section 6. 



2 Duration Calculus 



In this paper, the calculus we use is based on several variants of DC [5,6,4], but 
we shall still call it DC for short. 

DC was developed to reason about piece-wise continuous functions of time 
called states, which model the status of the system. In DC, time is represented 






102 



J. Gao and Q. Xu 



by non-negative reals. Intervals and interpretations of state variables are defined 
as follows 

Intv { [c, d] e Time G Time \ c e d} 

X G SVar Time ^ Values. 

Roughly speaking, a model is a pair (X, [c, d]) of interpretation and interval. A 
state expression is constructed from state variables with appropriate operators. 
A Boolean state expression, such as x < 1, with values represented by 0 and 1, 
denotes a property of the system. The duration that a Boolean state expression 
B holds in a model (X, [c, d] ) is defined as 

I J B]\(I, [c,d])^:^^ l'\[B]\{I,t)dt 

where |[ B ]| (X, t) denotes the value of B at time t under state interpretation X. 
The length I of an interval is dehned as 




and it is easy to prove 
1 1 ]|(X, [c, d]) = d (8) c. 

For any state expression S, we dehne its limit values at the beginning and the 
end of a non-empty interval as 

(b.^)(X, [c,d\)= \[S]\{I,t), 

(e.A)(X, [c,d]) ='^limt^^- |[S']|(X,t). 

A point interval is characterised hy I = 0, shortened as [] . A Boolean state 
expression B holds everywhere inside an interval and at a point are denoted 
respectively by \B] and and are formally dehned as 

(X, [c, d]) 1= [B] c < d and for any c < t < d, | i? ]|(X, t) = 1 

(X, [c, d]) 1= \B]^ c = d and | B ]|(X, c) = 1. 

Let \B]* \B] V \B]^. The modality ‘chop’ of ITL is dehned as follows: for 

any formulae A and B 

(X, [c,d])^A;B 

iff there exists m such that c & m E d and 
(X, [c, m]) 1= A and (X, [m, d]) |= B. 

Two modalities derived from chop are 
OA = true; A; true 
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A model satisfies OA and DA if respectively a sub-interval and all sub-intervals 
satisfy A. Both O and □ are contracting modalities in the sense that they only 
refer to sub-intervals. The following modality allows us to access intervals that 
are left neighbourhoods of the original interval 

(T, [c, d]) 1= 0[A iff there exists J G 0 such that {I, [c (A> S,d]) \= A 



The dual notion of right neighbourhood also exists, but we do not need it for 
our case study. 

The axiomatic system of DC includes that of ITL and the following axioms 
and rules. The axioms about durations are 

/0 = 0 

jBeO 

f B]_ + f Ba = f {Bi V Ba) + f (Bi A B 2 ) 

((/B = .T);(/B = y))^(/B = x + y) 

f Bi — f B 2 , provided Bi <tA B 2 holds in propositional logic. 

Mathematical theories can be imported into the calculus by the following 
MT rule 

Let R([II(S)~I ,b.S, e.S, 1) be a formula without chop and neighbourhood 
modalities. If in mathematics (Vc G d)R{'ic G f G d.H{S{t)), S{d~), 

d^S) c) holds, then [] V i?( [iL(S')] , h.S, e.S, 1) holds in DC, 



where S{c^) and S{d^) denote limits of S from the right of c and left of d 
respectively. 

The following are some examples of DC formulae 



(/ X > 1) < 0.25/ 

[x < 1] A / > 3 

([x < 1] A / > 3) 

; ([x > 5] A / = 1) 



duration that x > 1 holds is less than 
a quarter of the length of the interval 

X < 1 holds everywhere inside the 
interval and the length is greater than 3 

the interval can be divided in two parts satisfying 
respectively [x < 1] A / > 3 and [x > 5] A / = 1 



and theorems 

(fx < 1] A / > 3); [x = 0]°; ([x > 5] A / = 1) ^ [x < 1 V x > 5] A / > 4 
([x < 1] A / > 3); / = 1 ^ (/ X > 1) < 0.25/ 
h.y <5A|"x<lAy = x]A/<3^ |"y<8]. 



The proof system also contains two induction rules which are somewhat com- 
plex. In the last step of our case study, we need the following one 
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Backward Induction Rule. Let R{X) be a formula without the neighbour- 
hood operator. If R(M) and R{X) ^ R{Xy [\B\,X) V {\^B];X)) hold, then 
R(true) holds. 



3 Hybrid Programs 

3.1 The Language 

In [12], a language for describing sequential hybrid systems was proposed. The 
language supports constructive design by allowing specification statements to 
be mixed with program constructs, in the tradition of the well established the- 
ories of program development such as refinement calculus [2,9]. In addition to 
usual program constructs, the language includes a specification statement for 
describing (instantaneous) state transitions and a specification statement using 
a DC formula for describing a possibly hybrid component with time dependen- 
t dynamics. To describe concurrency, we extend the language with a parallel 
composition. 

The abstract syntax and informal interpretation of the basic commands arc 
as follows. 

P ::= {B{x) R{x,x’)) ]|] C ]| ] Pi\P2 1 Pi 0 P2 1 while 6 do P od 

S::^Pi II P2 II mill II Pn- 

The first statement is the guarded nondeterministic assignment: the process 
waits until the Boolean condition B holds, and then the statement is success- 
fully executed and effect is an instantaneous transition satisfying the binary 
predicate R. Sequential composition is represented by Pi\P2 and its meaning 
is that if Pi terminates then P 2 is executed immediately afterwards. Nondeter- 
ministic choice is represented by [] . Statement || C \\ describes a component 
by a DC formula C . Continuous evolution can be specified by an invariant ex- 
pressing the dynamical laws. Iteration statement is executed repeatedly until the 
boolean guard becomes false. Parallel composition is defined by the interleaving 
of component transitions. 

Other commands can be constructed from the above primitive ones, e.g.. 



X := exp{x) (true x' = exp{x)) 




1 . def 

skip — X X 




(B{x)) (true ^ B{x)) 




if Pi ^ Pi [] mill D P„ ^ P„ fi (Pi);Pi [] II 


nil D {Bn)-,Pn 



3.2 Verification of Hybrid Programs 

Verification methods for discrete programs, such as Hoare Logic, have been well 
established and can be found in text books (e.g., [1]). As an effective way to 
verify a hybrid program, it is suggested in [13] to use a program logic to verify 
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the discrete component, and incorporate the resTilts into a real-time temporal 
logic which is subsequently used to deduce properties of the whole system. For 
our case study, we extend the method to handle a limited form of interference 
that exists among processes. 

Sequential Discrete Programs In Hoare logic, a program S is correct with 
respect to a pair of pre- and post-conditions is denoted by 

M 

The meaning is that if the program is started in a state satisfying p, it will 
terminate in a state satisfying q. For example, the following is a valid correctness 
formula 

{x = 0} X := .X + 1 {.X = 1}. 

The following are some rules in Hoare Logic 

{P} >^1 >^2 {q} 

{p} Si]S 2 {q} 

{p} Si {g} {p} S 2 {q} 

{p} Si 0 S 2 {g} 

{pABi} Si {g} mill {pAB„} S„ {q} 

M if Hi ^ Pi [] ||||||[]p„^p„fi {q} 

t{i) a i > 0 ^ B r(0) ~^B 

{r{i) A i > 0} S {{3j < i)r{j)} 

{3i.r(z)} while H do 5od {r(0)} 

p^pi {pi} S {gi} gi ^ g 

W ^ {9} 

Proofs of programs are usually presented by annotating the program with as- 
sertions which serve as the pre-condition for the statement following it and the 
post-condition for the statement proceeding it. 

Linking Program Logic with DC We propose two rules linking program 
logic with DC 

{true} S |g(x)} 

S' II C II sch g(b.x) A C 

This allows the post-condition of a discrete program to be used as ‘pre-condition’ 
of the next phase. 

Ci^p(e.x) (p(^)} S (g(x)} 

II Cl II ;H;|| C 2 II sat Ci; (g(b.x) A C 2 ) 
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This rule says that the ‘post-condition’ of the first phase can be used as the pre- 
condition of the discrete program, and as in the rule before, the post-condition 
of the discrete program can be added as ‘pre-condition’ of the next phase. These 
rules allow us to reason the mixture of discrete and continuous components based 
on properties of the components. 

Soundness of the rules follows from techniques developed in [12,11], where 
the semantics of sequential programs is defined using a more involved extension 
of DC. The calculus is based on the Super Dense Duration Calculus first sug- 
gested by Zhou and Hansen [3] , and it can describe discrete transitions and their 
compositions. However, for practical applications, one can use ordinary DC and 
high-level rules like the ones presented above. For example, we have the following 
proof for the sequential program 

{true} 

X := 0; 

{x = 0} 

X := X + 1; 

{x = 1} 

y 0 

jx = 1 A y = 0} 

A phase which lasts for one time unit and obeys dynamic laws x = 0 and y || x 
is represented by|| [x = 0Ay|| x]AZ = l||.It follows that the hybrid 
program 

X := 0; X := X -f 1; y := 0; II [x = 0Ay|| x] A / = 1 || 
satisfies 

b.x = 1 A b.y = 0A[x = 0Ay|| x~\ Al — 1 
and we can further deduce for example that it satisfies 

[x = 1 Ay II 1] 



Concurrency and Interference In general, it is difficult to verify concurrent 
programs which may interfere with each other. But for particular applications, 
such as our case study, the degree of interference is usually limited, and con- 
sequently it is possible to use special and hence effective methods to verify the 
system. 

In our case study, the only shared variables are the sampled variables. They 
are read by the monitor and changed by the plant. The sampling action can 
be modelled by an assignment x := exp(yi, . . . , y„), where exp(yi, . . . , y„) is 
an expression involving variables yi,..., y„. If yi,...,y„ are continuous, then the 
Hoare triple 



{true} X := e.xp(yi, . . . ,y„) {x = exp(yi, . . . , y„)} 




Rigorous Design of a Fault Diagnosis and Isolation Algorithm 107 



is a valid axiom. However, if some of the variables may be changed discretely, 
as the fault functions in our case study, the triple is not a valid axiom, because 
the variables can change at exactly the sampling points. In most cases, it is 
reasonable to assume that the sampled variables can change at most once at one 
time point, and under this assumption, we can conclude that the sampled values 
are either the values before or after the transition if it indeed occurs. Formally, 
we have the following axiom 

{true} X := exp{yi , . . . , y„) (g) 

where q is the assertion {3zi , . . . , 2 „. x = e{yi , . . . , y„) A {zi — y^ \/ zi — yi) A 
. . . A {zn — y^ V Zn — y„)}> ill which Zi is a fresh variable and yF denotes the 
left limit of y,. 

The link rule can be generalised 
Ci^pje.x) |p(^)} -S' {q{xi 

II Cl II ;5';|| C 2 II sat (3ai, . . . , a„.(Ci A e.yi = oi A ||||||A e.y„ = a„) 

; (g(b..xi, . . . ,b.x^,ai, . . . ,a„) A C 2 )) 

Consider the simple hybrid program 

II Cl II ;x := := X > 0; || C 2 || . 

The system has two phases, and in between the value of y is sampled and after- 
wards u is set to true or false depending on whether the sample value is greater 
than 0. Variable y can be changed at most once by the environment at any point, 
whereas u is not changed by the environment. We have 

{true} 

X — y- 

{3z.x = z t\(z = y~ y z = y)} 
u X > 0 

{{u <(=> X > 0) A (x = V X = y)} 
and it follows that 

II Cl II ;x ~ y-u ~ X > 0; || C 2 || 
satisfies 

3a. (Cl A e.y = a); ((b.u AA b.x > 0) A (b.x = a V b.x = b.y) A C 2 ). 

Suppose Cl => Z = T and C 2 ^ I = T , that is, each phase takes T time units. 
Furthermore, assume C 2 stable(u), indicating that in the second phase u is 
not changed. We can prove by using DC that the program satisfies 

□ ((l>TAM)^O(y>0)) 

and 



□ ((1 > r A [y > 0]) ^ true; [u]). 
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4 System Analysis and Requirement Specification 

4.1 Sensors and fault models 

Under our model, the output of sensor i {i = 1, 2, 3,4) at time t is 

yi{t) = y{t) + i = 1,2, 3, 4, 

where y{t) is the nominal output, rii{t) is the noise function and fi{t) is the fault 
function. We have the following assumptions about these: 

— The nominal output is continuous with |y(t)| < L. 

— The noise is smaller than a positive constant value N. 

— Fault functions arc discontinuous: fi{t) — 0 when the *th sensor is not faulty, 
\fi{t)\ II F, where F is a positive constant when the ith sensor is faulty; faults 
occur instantaneously, that is, the values of fault functions can jump between 
the two kinds of values, but will not lie in anywhere between 0 and F. 

— Fault values are substantially larger than noise values, and in particular, we 
assume F > 4\\ N. 

Usually, sensors will stay in one status, either faulty or non-faulty, for a while 
before changing into a different one. We assume that there exist at least Ti time 
units between the changes of the status of any two sensors. Also, a sensor will 
not be faulty for a long time unless it is severely damaged. We assume that if 
a sensor has been faulty for tp time units, it must be caused by serious damage 
and the sensor becomes permanently faulty. 

We model sensor i as a hybrid program 

Ci := Q;cd := 0; fi := 0 ; 

while true do 

if Ci < tp A d > Fi A |/i| II F ^ d := 0; /* := 0;Ci := 0; c^ := 0 
0 Ci < tp A d > Ti A fi = 0 ^ d := 0; (true ^ |/'| || F); Cj := 0; Cd := 1 
[] true ^11 \[yi ^ y A m + fi) A fi ^ a Aui < N A d ^ I A Ci = Cd~\ || 

fi 

od 

in which Ci and d (initially d = 0) count respectively the time that sensor i 
has been faulty and the time since the last change of any sensor status. The 
change of the status of sensor i from faulty to non-faulty and from non-faulty 
to faulty is represented respectively by assignment fi 0 and nondeterministic 
assignment (true ^ |/(| || F). Continuous evolution of various variables related 
to the sensor is governed by laws in the phase statement. 

It is easy to prove that the following invariants are satisfied 

Property 1 ann,! || iY]*, nn/,! || F V /, = Oj ^ = 1, 2, 3, 4 
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Let 

1 \m\ ij F 

0 otherwise 

Boolean function Fi holds if and only sensor i is faulty. Since change of sensor 
status can only happen if the sensor has not been faulty for more than tp time 
units, ensured by the conjunct < tp in the guard, we have the following 
property 

Property 2 □(([Pil Al || tp);true^ \Fi]) 

Let 

F. = (/+ = 0 A /r = 0) V (I/+I II F A |/r| || F) 

AP 2 AP 3 AP 4 

Boolean function P, holds if and only if sensor i is not changing its status, and 
P holds if and only if none of the sensors are changing the status. The following 
formula expresses the property that between two changes, there is at least Ti 
units time 

Property 3 \P] \ ^ I || Ti) 

This implies the following lemma. 

Lemma 1 

/□(rp,l;(; II T);\F,-] ^ \F,]) 

II T); ^ hP,l) i = l,2,3,4 

Faults are typically independent, and in our model, we assume that if two 
sensors are faulty, the difference between the two fault functions are greater than 
a certain value G and G > 4 || A^. 

Assumption 1 

(/i / 0 A fj 7^ 0) l/j (g) /j I II G for any i, j e {1, 2, 3, 4}, i 7^ j. 

Under the normal condition, it is unlikely that three sensors are faulty, except 
the whole sensor area is damaged. We therefore assume that if there are three 
more faulty sensors, the fourth one is also faulty. 

Assumption 2 

^ 0 A ^ 0 A /fc / 0) ^ /fc ^ 0 
for anyi,j, S {l,2,3,4},i ^ j ^ k ^ 1. 
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4.2 Requirement 

Let 



V F,{t)AFj{t)AFk{t). 

i^j^k 

Boolean function FFF holds if and only there are three (actually four by As- 
sumption 2) faulty sensors. When this happens, it is impossible to use the sam- 
pled values in any way, and the final output will be assigned a default value FSV. 
Therefore, the required output is defined as follows 



Vrit) 



I FSV FFF(t) = 1 

[ y{t) otherwise. 



Due to delay in detecting fault occurrence, sometimes the actual output is 
different from the required one. Let denote the actual output and define 



_ / 1 \yv{t) yr{t)\ || e 

^^^10 \yv{t)^yr{t)\>e. 

Boolean function D holds if and only if the actual output deviates from the 
required one by not more than a fixed amount e. When D holds, we say that the 
system is under proper working condition. 

The first requirement is that the system should stay in proper working con- 
dition sufficiently often. More precisely, we require that if P holds initially for 
more than Ti time units, then the accumulated proper working time is not less 
than 100(1 ® 5)%(0 < 5 < 1) of elapsed time. This is expressed in DC as 



REQi = □(((PI A / > ri);true ^ 



D\\ (l®5)j|0. 



For this to be possible, the noise values cannot be too large. In particular, we 
assume 

Assumption 3 N II e/2. 

Permanent faults must be identified within a specified amount of time A. 
Let PFi be the boolean function which holds if and only if sensor i is detected 
to be permanently faulty. The second requirement is 

REQ2 □((Pil A (/ II tp + A)^ true; IFF,]). 

The third requirement is that if a sensor is considered to be permanently 
faulty, it must have been faulty for at least tp time units 

REQs □(! II tp; \PF,] ^ true; ([P,] A I || tp)). 
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5 Design of the Monitor 

The system is modelled as the parallel composition of the sensors, a monitor and 
the plant. 



SYS^Si II 52 II 53 II 54 II M/P 



Since the plant is very simple, we combine it with the monitor and denote them 
together by M/P. 

Our main job is to design the monitor so that the requirements are satished. 
As an initial attempt, we have the following framework of the system 



M/PP 

count{i) 0; PFi false {i — 1, 2, 3, 4); 
while true do 

{i = 1,2, 3, 4); 

if ~<PFi A ~'PF2 a ~'PF^ a ~iPF4 — > To; 

D i.i.fc.i PFi A^PFj A^PFk A^PFi P/; 

D i,j,k,i PFi A PFj A -^PFk A ^PFi P^^ \ 

0 PFi A PFj A PFk Vv ■■= FSV; 
fi 

II \{^FFF -^y, = y)A {FFF ~^yr = FSV) A |y| < Lj A 1 = T || ; 
PF^ PFi V count{i) || Tp (i — 1,2, 3, 4); 

od 



Variable count{i) records the time that sensor i has been detected faulty. Boolean 
variable PFi, as introduced earlier, indicates whether sensor i is considered per- 
manently faulty. It has the initial value false, and is set to true when the value 
of count{i) exceeds a thrash- hold value Tp, which is a control parameter yet to 
be chosen. 

Roughly, the system works as follows. At each point of sampling which has 
a cycle of T, the value of sensor i is sampled and stored in Xj. According to 
whether there are permanently faulty sensors, the following procedures Pq, P{ 
and P 2 decide if a sensor is faulty by comparing the sampled values. If a sensor 
is detected to be faulty, then the counter associated with it is increased at the 
end of the phase. During the phase, faults may occur, and consequently the value 
of requested output changes between the actual value y and FSV, with the 
latter being a default value. 
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Procedure Pq covers the case that there are no permanent faults. 

Po^ 

FQ (|xt ® Xjl II Th) A {\x^ Xk\ \\ A (|xj (g) x; | || T;,) (i = 1, 2, 3, 4); 

if 

^FCi A ^FC2 a —<FC^ a ~^FCi — > 

count{m) := 0 (m = 1, 2, 3, 4); := xi; 

D i,j,k,iFCi A -^FCj A ^FCk A ^FCi 

count{i) := count{i) + T; count{j) 0; 
count{k) := 0; count{l) := 0; := x^; 

D i,j,k,iFCi A FCj A ^FCk A ^ 

count{i) := count{i) + T; count{j) := count{j) + F; 
count{k) := 0; count{l) := 0; := x^; 

0 A FCj A FCfc ^ 

count{m) count{m) + T (m — 1, 2, 3, 4); FSV; 
fi 

By comparing the sampled value of sensor i with the values of the rest three 
sensors, it is decided whether it is faulty and variable FCi is assigned to true 
or false accordingly. This step is crucial and an appropriate thrash-hold value 
T/i must be chosen for the outcome of that decision to be meaningful. Based on 
the value of FCj, variable count{i) is updated, and output y„ is either assigned 
the value of a sensor which is considered non-faulty or the default value FSV if 
three sensors are regarded faulty. Procedures and P[^ are similar, and they 
are given in the appendix. 

As the design decision, the control parameters in the algorithm are chosen 
to satisfy the following 

/r,, = min(F,G)®2|| A^\ 

Fp= [^] ||F + 2||T DBS 

\T < min(f , ^)- / 

In the next section, we argue that the requirements are indeed satisfied. 



6 Verification 

We divide verification into several steps. First, we verify the control algorithm 
using program logic. Afterwards, we extract a number of properties and express 
them in DC. Finally, we prove that the requirements are implied by the estab- 
lished properties. For brevity, we will usually not write out all the conditions 
explicitly, but refer to them in the proof when they are used. We shall write P 1 
for Property 1, A2 for Assumption 2 etc. in the proof when there is not enough 
space. 
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6.1 Verification of Control Algorithm 

By the axiom in 3.2, we know that Xi = \/ Xi = yi [i = 1,2,3, 4) is a valid 

postcondition of the assignment Xi := yi. It can then be used as the precondition 
of Po- The following lemma shows that under this condition, if |x, ® Xj\ || 
Th A \xi ® Xfc| II Th A \xi ® x;| II Th holds, then either the left or the right limit 
of |/i| is greater than or equal to F, that is, sensor i has been faulty before or 
after the sampling point. 



Lemma 2 

[Xrn = y + n„i + /„) V (Xm = 2/ + + /“) m = 1, 2, 3, 4 

Wxi®Xj\\\ A |xj 0 Xfcl II Th^\xi®Xl\\\ T/J ^ /r 0 V /, yt 0)\ 
A(-i(|xi 0 xyl II A |xi 0 Xfcl II T^A|xi0xj||| Th) 

^ =0V/- =0)) J 

Proof: 

((^m — y ^ '^m, frn) ^ {^m — V frn)) ^ 

A(|xj0xy||| Th) for j 7 ^ i | {DBS} 

0 V /i / 0) 

f (|n, 0 rij + f\ 0 /y| II min(F, G) 0 2 || A \ 



V|r 



+ fi I II min(F, G) 0 2 || A 



V|n,0ny+/j 0/ylll min(F, G) 0 2 II A 
V|n, 0 rij + / “ 0 I II min(F, G) 0 2 || A) for j yt i 
Va/- =0A/, = 0 



{PI} 



7r =0A/, = 0 
A\fj I II min(F, G) 0 4 || A 

fj\ II min(F, G) 0 4 || A for j y^ i ^ 



|F > 4 II A and G > 4 II A} 



//r=0A/, = 0 
Afj yt 0 A 7 yt 0 for j yl i 

=> false. 



{A2} 



And 



/ {{Xm =y + rim + frn) V (x^ = ?/ + + /„)) m = 1, 2, 3, 4\ 

A-^dxj 0 Xjl II Tf, A |x, 0 Xfcl II rhA|xj0X(||| Th) 

V7/. = ov/-=o) J 

( {{Xm = y + nm + /m) V (x^ =y + nm + fm)) W = 1, 2, 3, 4\ 
A-^dxi 0 XjI II F?, A |x, 0 Xfcl II 7A|xj0xd|| 7) 

Afr II F A /r II F 

Va(|/-| II FV|/-| =0) for jy^f J 



{PI} 

{Al} 



/ {{Xm = y + rim + fm) {Xm = y + rim + fm)) m=l,2, 3, 4\ 
A-n(|xj0Xj||| F?, A |x, 0 Xfcl II Ff,A|x,0xd|| Th) 

A|/i 0 fj\ II min(F, G) A |/- 0 /r | || min(F, G) 

\ A|/“ 0 /j I II min(F, G) A |/“ 0 /F | || min(F, G) for j y^ i / 
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f ® Xj\ II Th A \xi ^ Xk\ || Th A |x, xi\ || Th) 
I A|xi ® Xj\ II min(F, G) (g) 2 || for j ^ i 
^ I ^{\xi ® Xj\ II Th A \xi g Xfel II ThA \xi <S)Xi\ II Th) 
YA|x'i ® Xj\ II Th for j 7 ^ i 

false. 



{DBS} 



Q, = (FQ A I/, I II F A |/-| II F) V (-FG, A /, = 0 A /“ = 0) 

The following diagram illustrates the relationships among Fj, FCi, and Fj. 



1 Pr 

0 .... 

1 F^ 

0 

1 FCi 

0 

1 Qi 



Time Axis 

Figure 2: The relationships among Ft, FCi, Qi, Pi. 

As shorthands, let 
Q — Qi A Q 2 A Qs A Q 4 

FFFe FCi A FC, A FCk,i,j,k e {1, 2, 3, 4} 

7(FG)‘'='A,((-FG,A(/, = 0V/-=0))V(FG,A(|/J II Fv|/-||| F))) 
I{count) /\^{{^FCi A count{i)= Oy) V (FG, A count{i) = count{i)^ + T)) 
I {FC, count) I {FC) A I {count) 
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We have the following proof outline for procedure Pq: 

{^'m Vra ^ 1/m} 

FC^ ■.= {\xi® Xj\\\ Th) ^(\xi'Sl Xk\\\ Th) /\(\xi® xi\\\ (i = 1, 2, 3,4); 
/ {Ftti — Vm ^ — ym)^ 1 

\ (FCi ^ (Ixj ® II Th) ^{\xi®Xk\\\ T/i) A (|xi ® a;;| II Th)) } 

{{Xm = y^n V = I/m) f\I{FC)} {Lcmma 2} 

if 

~^FC\ A ~<FC2 a ~^FC^ a ~^FCi — > 

{{Xm — Vm'^ — ym) A I{FC) A ^FCi A -iFC2 A ^FCs A ^FC4} 
count{m) := 0 (m = 1, 2, 3, 4); := xi; 

f (a;m = ym'^ ym)^ \ 

^ I[FC, count) A ~>FCi A ~^FC 2 A ~>FC 3 A ^FC^ A = Xi J 
[] i,j,k,iFCi A A -^FCk A ^ 

{(a^m = llm V Xm = 2/m) A I{FC) A FCi A -'FC'j A ~^FCk A ^FCJ 
count{i) count{i) + F; count{j) 0; 
count{k) 0;count{l) := 0;y„ := x^; 

J {Xjyi = ^ ~ Vm)^ 1 

^ I{FC, count) A FFi A ~<FCj A ~'FCk A ~^FCi A y^, = x^ J 
D i,j,k,iFCi A FCj A -^FCk A ^FQ ^ 

{(a:™ ^ y:^y Xm ^ ym) A I{FC) A FQ A FCj A -'FC^ A -iFC;} 
count{i) count{i) + T; count{j) count{j) + T; 
count{k) 0;count{l) 0;y„ := Xfc; 
f (a'm I/m a^m ym)^ 1 

\ I{FC, count) A FCi A FC^ A ^FCk A ^FC; A y„ = Xfc j 
D i,j,kFCi A FCj A FCk 

{{Xm ^ ym'^ Xm ^ ym) A /(FC) A FCr A FCj A FCk] 
count[m) := count{rn) + F (m = 1,2, 3,4); y„ := FSV; 

{(a:m = 1/m V Xi = yi) A /(FC, count) A FCi A FCj A FCfc A y^, = FSV} 

fi 

{/(FC, count) A (Q A -^FFFc => |y„ <E> y| || A^) A {FFFc {yv = FSV))} 

The fact C A -iFFF^ ^ |y^• <8 y| || is a valid post-condition relies on the 
following lemma 

Lemma 3 g A (xfe = y^ V Xfe = yk) A -■FCfc => |xfc ® y| || 

Proof: 



Q A {xk = y]] Xk = yk) A -iFCfc 
^ (a^fc ^ yk y Xk^ yk) A = 0 A /fc = 0 
^ Xfc = y + nfc 
^ |xfc ® y| II N 



(Definition of g} 
(Definition of yk] 
(Property 1} 



□ 
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We can similarly prove the same postcondition for procedures PI, P 2 ■ It is 
easy to see that we can associate the following proof outline for M/P 

M/P 

count{i) := 0; PFi := false {i = 1, 2, 3, 4); 
while true do 

Xi (i = 1,2,3, 4); 

{{x^ = y// V X, = yi) [i = 1, 2, 3, 4)} 
if —<PFi A —<PF2 a —<PF^ a ^PF^ — > Pq', 

D i,j,k,i PFi A ^PFj A A ^PFi — P/; 

D i,j,k,i PFi A PFj A -^PFk A ^PFi P^^ ; 

D i,j,k PFi A PFj A PFk yv FSV; 
fi 

{I{FC, count) A (Q A -^FFF^ ^ ® y\ |1 N) A {FFF,,^{y„ = FSV))} 

II \{-.FFF ^yr=y)h {FFF ^ y, = FSV) A |y| < L) A / = T || ; 

PFi := PFi V count{i) \\ Tp (i = l,2,3,4); 

od 

We next incorporate the results into DC, based partially on the link rules and 
partially on operational reasoning. The remarks following the stated property 
give some explanation why it holds. 

Property 4 □([Fi] Al >T ^ I \\ T; \FCi~\) 

Remark: If F) holds (i.e., sensor is faulty) for a period of time greater than T, 
then at the next sampling time, this will be detected with FCi set to true. 

Property 5 □(! || m || T A \FCi] ^ 0(|"Fi] A ^ || (m (8> 1) || T)) 

Remark: If FCi holds for greater than or equal to m || T time units, then Fi 
holds either in the left or the right neighbourhood intervals of the first m || T 
sampling points (including the beginning point of the interval if it starts with a 
sampling point). Since the distance between two neighbourhood intervals is less 
or equal to T and T < 7}, it follows from Lemma 1 that Fi holds everywhere 
between the m sampling points. 

Property 6 □(/ > T A \P) => / || T; [Q]) 

Remark: If the status of a sensor is not changed, then after a sampling point, 
the detected status is the actual one and this is maintained in the rest of the 
interval. 

Lemma 4 

^({\\yv^y\ II A^l°;true) A [yr = y A |y| < L] Astable(y^) A (f II T)\ 

Hyr ® y,;| < el ) 

where stable(y„) denote variable y^ does not change in the interval. 
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Proof: 

(riy«®ylll iVl°;true) A [j/r- = 2 / A |y| < L] Astable(y^,) A (/ II T) {A3} 
^(n?/«'»ylll fl°;true) A [7/r = y A |y| < L] Astable(y„) A (Z II T) {MT} 
^ WVv <8> y| < f + i II T"! A \vr = y] (T < 

^ [lyf ® yr I ^ f] 

□ 

/□([Q A -iFFFc] ^ riy^; ® yr| II el) 

Property 7 I A 

Vn(rQ A FFFel ^ \y, = Vr = FSV]) 

Remark: It follows from Lemma 1 and the Lemma 4 above. 

Property 8 □([FC'.;] ^{l> T^) (true; [FFj])) 

Property 9 D{\PFi'] => Oi{\FCi] A I || Tp)) 

Remark: Since I (count) holds before the phase is entered, it follows that if 

FCj holds at the sampling point, then the associated counter will be increased. 

Therefore, if FCj holds longer than Tp, then the counter will reach the value Tp 

causing PFi to be true. On the other hand, if FFj is set to true, FQ must hold 

longer than Tp. 

6.2 Verification in DC 

We now deduce requirements REQi, REQ 2 and REQs in DC. 

Lemma 5 □([FC',] Al|| Fj, ^ (true; ( [Fj] A 1 || tp))) 

Proof: 

(FQl A 1 II Tp {Tp = [^] II F + 2 II F, Property 5} 

^ 0([Fi] A (I II [■^] II F + F)) (Arithmetics) 

^ true; ([Fj] A / II tp);true (Assumption 2} 

^ true; ([Fj] A I || tp) 

□ 

/□((F,lA(/|| tp + A)^(true;rPF,l)) 

Theorem 1 A 

\d({1 II tp;[FF,l)^(true;(rF,l Al|| tp))) 



Proof: 

\P]Al\\tp + A {T<f} 

^\P] A (I > [^1 II F + 3 II F) (Property 4} 

^(/|| F;}F,AFC,1)A(;> [|]||F + 3||F) (DC) 

II F; (\Fi A FCi] A(l > [^] || F + 2 || F)) (Arithmetics) 

^true; (\FCi] A / || Tp) (Property 8} 

^true; [FF*] 
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And 



I II tp; [PF,] 

Al \\ Tp) 

II tp-Oi{Uue;\Fi] Al \\ tp) 
^true; ( [Fj] A Z II tp);true 
^true; ([Fj] A I jj tp) 



{Property 9} 

(Lemma 5} 

{DC} 

{Property 2} 

□ 



Lemma 6 □({Q A F] => \D]) 

Proof: 



[Q A F] {Definition of F and Q} 

^\Q A -iFFFc] V [Q A FFFc] {Property 7} 

=^\\yv ® l/r| II el V \\yy ®yr \ =0] {Definition of D] 

^\D] 



Theorem 2 



Proof: 



/n{\P]Al\\ Ti^ I <5 II 1) 

Vn(({FlAMI ri);hFf;ri"l ^/^D<^||0 



□ 



[F] A / II Ti 

^{FlAMI T,A[1 II F;{Q1) 
Ml II T;rQAFl)AMI Ti 
Ml II T;\D))AI\\ Fi 
MI^D II T)Al\\ Fi 
< Sjl I 



{T < Property 6} 
{DC} 
{Lemma 6} 
{DC} 
{T < ^} 



And 



([FI ami Ti);h^r;lFl 

{l\\ T;{gAFl;hFf;(M| T 'V {I || T;{QAF1))) 



\^i II n 

^(/|| T;\D];l\\ T ; {\) W \ D))) A I \\ T, 
MI^DW 2 \\T)aI\\ Fi 
j ~^D < (5 II / 



{P6, DC} 

{L6} 

{DC} 
{F < ^} 



□ 



Theorem 3 □(( [F] A I || Fi); true ^ f -<D < |M) 

Proof: We need to use the induction theorem. Let 

F(A) ='n(([Fl ami Fi);h^"f;rF];A^ j^D<5\\l) 
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It follows from Theorem 2 that ^(H) holds. We next prove 



R{X) ^ R{X V i\P];X)V i\^P];X)). 



Because 

R{Xy{\P];X)V{\^P];X)) = false} 

^ R{X V i\P]:X)) {Definition of i?(X)| 

^ □(({FI A I II Ti); h^f ; [FI; \P];X^J^D < 6 || 1). 

It is sufficient to prove 



F(X)^(([F1 A/|| Fi);hFf;[Fl;[Fl;X^ | || /) 



({FI ami ri);h^’f;rFl;{Fl;X 
({FI A Ml Ti); {^Ff; [F] ; ( {^Ff V {Ff ); {Fl;X 
(({FI ami Ti); {-Ff ; {F{; {-Ff ; {F];X)' 
V(({F]AMI ri);{-FlO;{F];{FlM{Fl;X) 



{DC} 

{DC} 

{P3, DC} 



A({F]AMI Ti);{^Ff;({Fl ami F); {^Ff ; {F{ ; X) 
1,V(({F1 aMI Ti);{^Ff;{F];X) 



{T2, R{X)} 



f{{J^D<S\\l)-\^Pr;{J^D<S\\l)) 
\y J^D<6\\1 



{DC} 



^ f < 5 \\l. 



Therefore, F(true) holds, that is 

□ (({FI A I II Ti); {-Ff ; {Fl;true ^ J < 6 \\ 1). 
It follows that 

({FI ami Ti); true {DC} 

fi\p]^i\\ n) 

^ V(({F 1 ami ri);{-Ff;{Fl) 

Vv(({Fl ami Ti);{-Ff;{F];true) 

^ [ —<D < 5 II L 



□ 

It follows from Theorems 1 and 3 that the requirements are satisfied. 

7 Discussion 

In the paper, we have presented a rigorous design of an FDI algorithm. Such 
systems are quite complex, and we have used two techniques, namely. Dura- 
tion Calculus and program logic, to verify the algorithm. Our verification is not 
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completely formal, in that operational reasoning has been used at a number of 
places to incorporate the results of program verification into DC. More research 
is needed to provide a more formal link between DC and program logic. 

We have simplified the fault model in several aspects. For example, we assume 
that when two sensors are faulty, their outputs will be larger than a constant G. 
In practice, this may not necessarily be the case, but only with a high probabili- 
ty. Therefore, a more precise model would include probability, and more research 
is needed for reasoning about such systems. 



Acknowledgements We are grateful to Zhou Chaochen who has been involved 
in the requirement analysis and the modelling of the system. Paritosh Pandya 
checked some lemmas using his validity checker for DC and found a mistake. 
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Appendix 

Procedure P{ handles the case that only sensor i is permanently faulty. 

P{^ 

FCj ■= {\xj ®Xk\\\ Th) C{\xj ®xi\\\ Th) 

if 

[] i,k,i^FCj A -^FCk A -^FCi 

count{j) := 0; count{k) := 0; 
count{l) := 0;yv '■= a^fc; 

[] j,k,iFCj A ^FCk A -'FC'; ^ 

count{j) := count{j) + T\ countfik) := 0; 
count{l) := 0;yv ■= Xk\ 

[] j,kFCj A FCk countfij) := count{j) + T; count{k) := count[k) + T; 
count{l) := couTit{l) + T;yy := FSV; 

fi 

Procedure P[^ handles the case that sensor i and j are permanently faulty. 

P^ = 

FCk ■■= \xk<S'Xi\ \\ Th {k,l^i,j)- 

if 

[] k^FCk count{k) 0;count{l) 0;y„ := Xk', 

[] kFCfc count{k) := countfik) + T; courit{l) := count{l) + T;yy := FSV; 

fi 
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Abstract. We address the problem of synchronizing estimates of plant 
state. The approach proposed in the study is unique because it docs not 
attempt to transform the data to a common representation. Rather we 
establish a framework which we call the Multiple Agent Hybrid Estima- 
tion Architecture in which we allow heterogeneous data to flow between 
individual agents in the network to improve their individual estimates of 
the current plant state. 



1 Introduction 

This paper is a brief outline of our multiple agent hybrid estimation architecture 
(MAHEA) and its variational basis. It is designed to model fusion of heteroge- 
neous sensor data and also to model coordination of heterogeneous sensors for 
distributed plants. It can be used for virtually any application in which sensor 
fusion of heterogeneous sensors is required. Most sensor fusion strategies may 
be mathematically recast as synchronization problems in the relaxed calculus of 
variations. The MAHEA architecture is based on our MAHCA architecture and 
is specifically targeted at hybrid system applications in which one has to extract 
digital programs for coordination and control of heterogeneous sensors. In the 
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battlefield and for large economic and industrial systems, there is no central su- 
pervisor with complete information about the dynamics of all parts of processes 
in the system. MAHEA models these processes as a distributed system in which 
there arc semi- autonomous agents, each sensing parts of certain processes. Each 
agent has information about the measurements made by its sensors. Each agen- 
t has only partial information about measurements made by sensors observed 
by other agents. This partial information is communicated by those agents via 
communication lines. MAHEA is an adaptation of MAHCA, our Multiple Agent 
Hybrid Control Architecture ([18], [16], [14], [9], [19]), to estimation problems. 
MAHCA itself was designed to extract digital control programs for a variety of 
processes including automated manufacturing, multimedia networks, controlling 
flexible gun tubes, flight planning for missiles, battle management, traffic man- 
agement of highway systems, supply line planning and logistics, etc. MAHEA is 
more specifically designed for sensor fusion, distributed simulation, and coordi- 
nation and synchronization of semi-autonomous units in any large business or 
industrial enterprise. We allow heterogeneous data to flow between individual 
agents in the network to improve their individual estimates of the current plant 
state based on their internal model of the dynamics of process states and of the 
sensors that sense the process states. We provide a brief description of a MAHEA 
agent, the basics of a MAHEA agent model of the plant, and the procedure it 
uses to improve its plant estimate over time. Each agent of MAHEA formulates 
a relaxed variational optimization problem whose successful resolution produces 
an estimate of the plant state. Each agent operates as a real-time theorem prover 
in the domain of relaxed variational theory. Its purpose is extraction of real time 
estimates of plant states and real time coordination of plants in a distributed 
setting with no central supervisor and incomplete information. The MAHEA 
architecture is based on the construction of estimation Lagrangians on a mani- 
fold designed for the problem at hand. In the dual Hamilton- Jacobi formulation, 
the problem is a relaxed dynamic programming problem on the manifold for 
computing geodesic fields. We have developed a variety of differential geometric 
techniques for solving such problems using connections, covariant derivatives, 
and Cartan derivatives associated with the Hessian form for the corresponding 
Lagrangian. This is not discussed here. First we give a basic mathematical model 
for sensor fusion. There are two fundamental problems addressed by a MAHEA 
agent. 

Fundamental Problems. 

1. Estimate the evolution of global plant state from the evolution of heteroge- 
neous sensor readings. 

2. Develop mechanisms for coordinating sensors which operate in different local 

clock-space coordinates so that they provide consistent estimates. 

An example of 2. is the problem of synchronization of the Army’s nation 
wide distributed simulation net. Some of the sites interacting in single theatre- 
wide simulations are real army-fielded units consisting of men and equipment 
in a real environment, some sites are purely battlefield simulators, some sites 
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are real personnel interacting with simnlated vehicles, some sites are simulated 
personnel interacting with real vehicles, etc. One does not wish to shoot down 
a plane at one site in local time, and then at another local time and site, be 
sensing it as continuing to fight, due to communication delays. The Army sim- 
ulators that are supposed to interact are of different generations, using software 
and hardware written in incompatible languages under incompatible operating 
systems on incompatible machines of uncertain age, with entirely different time 
and space granularities in the models. 



2 Sensors and Measurement Maps 

Sensors. Suppose the evolution of the time history of a plant is being observed 
by a family of heterogeneous sensors. What is observed might be the evolution 
of motion of a car or airplane, the evolution of the state of a battlefield, or 
the evolution of the state of a factory, or business. Each sensor gives sensor 
readings as a function of time and of the state of the plant. That is, each sensor 
determines a map from the space of plant states to the space of sensor readings. 
The observed plant always has partially unknown dynamics. One never succeeds 
in exact measurement of the state of an actual physical plant. All that is ever 
available is a lot of simultaneous readings of states of sensors. 




I I I 

Sensor Agents 

1 1 1 



t 

o 

t 



Interagent 

Communieation 



Sensors 




Fig. 1. Agent framework 



In our multiple agent sensor estimation architecture, which is an instantiation 
for multiple heterogeneous sensors of our multiple agent hybrid system architec- 
ture, we equip each sensor with a software program, called a sensor estimation 
agent. This agent has internal to its program an executable model of sensor 
dynamics for the scnsor(s) for which it is the agent. It also has a Lagrangian 
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executable model of plant dynamics. It is equipped with communication links 
to (some) other sensor agents. These links transmit information about the local 
sensors and the plant to some other sensors over a communications network. 
For instance, if sensors and their agents are monitoring a battlefield, their a- 
gents may be providing to other agents the results of video, radar, or infrared 
readings. See Figure 1. 

Sensors as smooth maps. Each sensor gives sensor readings as a function 
of time and of the state of the plant. That is, each sensor determines a map from 
the space of plant states to the space of sensor readings. We model the plant 
states as a smooth compact finite dimensional manifold M . We model each sensor 
as a smooth map s : M —> W from the manifold of plant states to the reals. Such a 
map is usually called a scalar of the manifold. If a sensor produces measurements 
of k quantities Sj{x), we regard the sensor as defining a vector of k scalars; the 
sensor then maps the manifold of plant states into /c-dimensional Euclidean 
space. We remain vague as to what “smooth” means. As is usual in applied 
mathematics, “smooth” means whatever is required in the immediate argument. 
Sometimes smooth means twice continuously differentiable; sometimes it means 
C°°; sometimes it means analytic; whatever is required for the computation at 
hand. 

Example. A fixed position black and white video camera transforming the 
observed scene into a c row d column pixel sereen with intensity at a pixel 
as value. This defines cd scalars, one for each pixel position, the value at the 
pixel position being the intensity of the signal, and the vector values are in cd- 
dimensional Euclidean space. 

Representation of Concrete Sensors. We obtain the smooth map for 
each sensor in a concrete problem from the differential equations governing its 
input output relation, equations obtained from the radar, infrared, video camera, 
etc. designers and manufacturers. For our MAHEA architecture the output of 
each sensor is treated by its agent as arising from a geodesic out of a geodesic field 
for a Lagrangian variational problem obtained from the differential equations for 
the sensor by an inverse variational method. 

The variational approach to sensor fusion here differs from other approaches 
to sensor fusion of which we are aware in that it does not attempt to transform 
the instantaneous readings of heterogeneous sensors to an instantaneous estimate 
of plant state, but rather transforms past evolution of readings of heterogeneous 
sensors to estimate present and future evolution of the plant state. 

The measurement map and measurement manifold. Simultaneous 
sensor readings by all sensors of plant state give simultaneous values of the 
associated scalars s^ : M ^ M on the plant state manifold. If k is the total 
number of scalars, this defines a measurement map M : M ^ given by 
M(x) = (si (x), ..., Sfc(x)), of the plant space M onto the measurement space 
N = M(M) II We assume W is a smooth compact submanifold in See 
Figure 2. 




126 



W. Kohn, A. Nerode, and J.B. Remmel 



plant space measurement space 
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Fig. 2. Sensor fusion - single agent 



For concrete sensors, we can compute the measurement map M from the 
physical equations governing the sensors as a vector of the input-output maps 
of the sensors. For radars, infrared sensors, video cameras, etc., these equations 
are standard and readily available from the designers. But they seem not to 
have been previously used simultaneously for plant state estimation, that is for 
sensor fusion, or for synchronization. The measurement map M tells how the 
plant state is transformed into a measurement vector. What we normally observe 
is not the trajectory of evolution of plant state on the manifold of plant states, 
but rather we observe the evolution of the image trajectory of measurement 
vectors in the measurement manifold. The measurement map concept gives a 
geometric meaning to sensor fusion. 

The Algebra of Scalars. If we are given a compact Hausdorff space 
M, and C{M) is the Banach algebra of continuous real valued functions on M, 
then by Urysohn’s lemma the “measurement map” I : M (g>-^ given 

by I{x){f) — f{x) is 1-1. In the interpretation offered, I{x) would represent the 
result of simultaneously reading the output of every possible sensor (continuous 
scalar) at state x. This says that if we had a sensor for every continuous scalar, 
we could recover the plant state x from all the sensor readings of x. Much more 
is true. 

The Gelfand-Mazur theorem implies that when M is any compact Hausdorff 
space, M is naturally homeomorphic to the spectrum of C{M), that is, to the 
maximal ideal space of the Banach algebra C{M) ([27], p.l23). There is a sim- 
ilar result for C°° manifolds M. There C{M) is replaced by C°°{M), and the 
spectrum of carries the C°°-nianifold structure. 

Any measurement map M arising from a finite set of sensors (scalars) may 
be regarded as a finite approximation to the embedding I . 
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Rank n measurement map. Suppose the plant state manifold M is n 
dimensional. When is a set of sensors enough to recover a point x of M from 
its image under the measurement map M? If in a neighborhood of a point x 
of M , the rank of M is n, then the inverse function theorem says that locally 
around x, we can recover the plant state trajectory 7 in M from its image 
trajectory M ||7 in the measurement manifold N . This is an algorithmic recovery, 
if the measurement map is constructive and smooth, so that the inverse function 
theorem can be constructively applied. 

Rank < n measurement map. Suppose that in the neighborhood of 
point X, the rank of M is less than the dimension n of M . How closely is the 
plant state trajectory at x determined by the measurement manifold trajectory 
through its image M(x)? The derivative of M at plant state x is the Jacobian 
transformation mapping the tangent space at x in M to the tangent space of the 
measurement manifold N at M(x). If we think of plant state trajectory evolution 
as defined by the direction field of its state trajectory, then knowing the direction 
of the image trajectory through M(x) determines the direction of the pre-image 
plant state trajectory on M through x only up to adding any vector in the kernel 
of the Jacobian of M at x. This is the natural infinitesimal limitation on recovery 
of direction of a plant state trajectory through x from the image direction under 
the Jacobian of M. When the kernel has dimension > 0, there arc d “degrees 
of freedom” in the determination of the plant state trajectory direction from the 
image measurement trajectory direction. Otherwise put, the measurement map 
image secs the tangent plane at x mod the kernel of the Jacobian of M, that is, 
it sees an n(S)d dimensional picture rather than the true n dimensional picture. 



3 Multiple Sensors and Mnltiple Agents 

The discussion above fits the situation where there is a single agent with perfect 
information about all sensor readings all the time, who can process this infor- 
mation instantaneously. In the battlefield and for large economic and industrial 
systems, there is no such central reservoir of complete information. These are dis- 
tributed systems in which each agent has information about the measurements 
made by those sensors that that agent observes and only partial information 
about measurements of other sensors observed by other agents. We adapt our 
Multiple Agent Hybrid Control Architecture as a Multiple Agent Hybrid Esti- 
mation Architecture (MAHEA). We allow heterogeneous data to flow between 
individual agents in the network to improve their individual estimates of the 
current plant state based on their internal model of the dynamics of the plant 
state manifold. See Figure 3. 

The effect of simultaneous measurements. It is important to note that 
simultaneous measurements by sensors each of rank < n can be used to construct 
simultaneous measurements maps of rank n, allowing local recovery of the point 
X from its sensor measurements. For example, suppose n — 3 and we have two 
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measurement spaces 




Fig. 3. Sensor fusion — multiple agents 



agents i — 1,2, with their measurement maps : M ^ on a compact 
manifold M || from scalars Sij for j = 1,2, with each of rank 2. Then 
the simultaneous measurement map M : M ^ into the product space 



M(xi, X2, X3) = (sii (xi, X2, X3), Si2(xi, X2, X3), S21 (xi, X2, X3), S22 (a:;i , ^2 , X3)) 



may well have rank 3, thus allowing local recovery of the plant state (xi,X 2 ,X 3 ) 
from the measurement map image M(xi, X2, X3). This is one standard way to 
recover three-dimensional images from two-dimensional images from disparate 
sources, such as infrared, radar, or video shot from different locations. 

Incorporating time. The model above needs to be supplemented for sen- 
sors which are not physically fixed relative to the observed manifold. If sensors 
are moving relative to a changing plant manifold, we need to introduce a time 
coordinate, extending the manifold, the measurement map, and the sensor equa- 
tions by a time dimension. 
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4 Conservation Laws and Elimination Methods 

In our MAHEA architecture, we use observed deviations from conservation laws 
about measurement map values y to correct by feedback the sensor and plant 
models, parameters and initial conditions. How does one compute such conser- 
vation laws? In the variational formulation, conservation laws are obtained by 
the Noether algorithm. In the case of sensor fusion, the analogous method based 
on the same principles uses symbolic calculus to eliminate x from the cquation- 
s for sensors which involve input x, and output y to get differential relations 
on y alone, possibly infinite in number. These express the desired conservation 
laws. These are equations among coordinates y — [yi, .... ym) of the value of the 
incasurcmcnt map M which must hold at all times if the measurement values 
y come from at least one possible manifold state x. Any deviation from such 
conservation laws between the coordinates of y means that there is no possible 
X in the manifold M for which y is the measurement vector. That is, there is no 
possible state of the manifold as observed which could give rise to these mea- 
surements. This has to be traced back to incorrect or unmodelled dynamics in 
the description of the sensors, whether differential or variational. 

There are many other elimination methods which may be of help for finding 
conservation laws for limited classes of problems. In case the system of differen- 
tial equations in x, y from which x is to be eliminated is a system of algebraic 
differential equations, built up by rational operations and differentiation from 
variables, then Ritt’s elimination procedure for differential algebra (alternate- 
ly, the decision method for the theory of differentially closed holds) says that 
there is an algorithm which gives a set of conservation laws for the y which arc 
necessary and sufficient for the existence of x such that (x, y) satisfy the equa- 
tion. Thus in this case there is a set of conditions which can be computed and 
which are met by y iff the measurements arise from some plant state x. This is 
a kind of vanilla generalization of Noether’s algorithm (and its converse since 
the existence of measurements y satisfying the system with x eliminated imply 
that the original (x, y) system is satisfiable by some plant state x.) In case the 
transformations are real (complex) algebraic, there are the elimination methods 
for real closed (algebraically closed) fields of Tarski. There are recent results on 
definability in o-minimal structures and hybrid systems which may prove useful 
if they can be improved enough to exhibit elimination of quantifiers in terms of 
understandable basic predicates. 



5 Noether Synchronization of Agents 

Geodesic Fields. We employ Caratheodory’s definition ([5], [25], p.l88). On 
a bounded region A || M of an n-dimensional manifold M, a geodesic field for 
a Lagrangian L is defined by: 

1. a set of n C^-functions (f>j[t,x) defined on the product P = [ro,ri] || A of a 
hnite closed time interval and the region A, such that the integral curves of 
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X = for j — 1, yield a family of curves which cover P simply; 

and 

2. a C^-function S{t, x) with the same domain P such that if a new Lagrangian 
is defined by 



L*(t, x,.x) = L(t, X, i) ® — 0 - — Xj (1) 

at oxj 

(which obviously has the same geodesics as L in the region) then L*{t, x, x) = 
0 whenever x = (j){t,x), and L*{t, x, x) > 0 otherwise. 

The existence of a geodesic field and det ^ dxdi ^ ^ ^ implies that the Weier- 
strass excess function is non-negative: 



E{t, X, 4>, x) = L{t, X, x) <8) L{t, X, (/)) ^ ^ II 0 (2) 

OXj 

Then a corresponding S can be computed from the Hamilton- Jacobi equation 
If + If ) = 0 where P = |f , and det / 0 implies x = (f>{t,x,p), 

and the Hamiltonian is defined by 

H{t,x,p) = (S)L{{t,x,(l){t,x,p)) +p4i{t,x,p) (3) 

(see [25], pp. 194-198.) 

We assume that each agent is an estimation agent, estimating the current 
state of a manifold x at time t. This may be in applications a measurement mani- 
fold rather than the original plant state manifold. In accordance with the general 
MAHCA approach we assume that the model of state evolution is expressed so 
that evolution of state is a geodesic - i.e. satisfies the Euler-Lagrange equations 
- corresponding to a non-negative Lagrangian cost function. Each agent has a 
Lagrangian, and is estimating state evolution as a geodesic according to his or 
her own Lagrangian. The agents, whose .sensors observe the same manifold, may 
have quite different Lagrangians. (Think of two radars observing the same field, 
or of a radar and an infrared video camera observing the same field. Each gives a 
current pixel image estimate. Wliat is the relation between tlie images?) We will 
assume that each agent has a Lagrangian with a given geodesic field in the sense 
of Caratheodory, say over the same region, but based on different Lagrangians. 
What definition can we give for synchronizing their states? Suppose that the 
local time and state for agent 1 is (ti,xi) (state-clock coordinates). Define a 
(local) synchronization of state-clock coordinates of agent 1 with those of agent 
2 as a smooth function 'I'{x, t) such that in a neighborhood of (H, Xi), we have 

Li{t,x,x) = L2{'I't{x,t),E^{x,t),4'^{x,t)) (4) 

This means that one side is zero if and only if the other is zero, and according 
to the definition of geodesic field, one side is greater than zero if and only if the 
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other is. Thus in the domain involved, [x, x, t) is on the geodesic field for agent 1 
if and only if t), 'Pxix, t)) is on the geodesic field for agent 2. The 

meaning of t) is that if agent 2 knows 'P and her own state-clock coordinates, 
she knows corresponding state-clock coordinates for agent 1 in the sense that 
costs agree along corresponding geodesics. We are assuming that cost plus being 
on the manifold is all that counts. Similarly if agent 2 knows a synchronization 
4?, such that 

L 2 {t,x,x) = Li{<Pt{x,t),<Px{x,t),<i>x{x,t)) (5) 

in a neighborhood of local state and time, he knows the corresponding time, 
state and direction that gives agent 1 the same costs along her geodesics. So for 
our form of synchronization and coordination, we need to know how to compute 
local synchronizations. This is where the Noether algorithm comes in. 

To illustrate how this method is used, suppose that there is a global La- 
grangian state estimation function for the plant L{t, x, x) and we have two a- 
gents, Agent 1 with its estimation Lagrangian Li{ti,xi, ,x\) and Agent 2 with 
its estimation Lagrangian ^ 2 (^ 2 , T 2 ). Moreover assume (in order to get im- 

plicit equations for them) that we have state and clock transition functions for 
Agent i — 1,2, given by: 

x\ = xl{t,x,w), i = l,...,n .g, 

ti = ti{t,x,w) 

where w = {w^, . . . , w") is a set of auxiliary parameters. Embedding in a smooth 
family with a auxiliary parameter w is the basis of Lie’s and Noether’s method of 
finding symmetries for differential equations and variational problems. The tc* 
have a simple meaning in the sensor interpretation. They are the sensor readings. 

Thus is the rate of change, or sensitivity, of state variable Xi with respect to 
the k ®th sensor reading. We want to determine x\ {t, x, w), ti{t, x, w), solving 

L^{xt{t, X, w), Xi{t, X, w), ±i{t, X, w)) = L{t, X, x) (7) 



for z = 1,2. 

Next we state the Noether Invariance relations. Define two classes of infinites- 
imal transformations for each Agent i: 



and 



dxi 

dw^ 



a=o = OlkiU,x) 



dtj 

dw^ 



|uJ— 0 — X 



( 8 ) 

(9) 



Next let Eij, for i,j = 1,... ,n, be the Euler-Lagrange operators for Li. {see 
[25], p.l85) That is, let 
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This given, the Noether Invariance Relations (see [25], pp. 201-207) are given by 



dli 



^ ^ I (11) 

i=i V i=i 



for k = 1 , . . . ,n 

If the estimation Lagrangian has an extremal, i.e. if there exists a curve 
Xi{t) = 7i(t) which over the usual admissible family of curves minimizes 






{t% ; Xi , 



( 12 ) 






where 7i(to) = xq and 7i(to + ^) = x±, then Li must satisfy the Euler-Lagrange 
equations E^ j{Li) — 0 for j — . , n,;. In that case the left hand sides of (11) 

are 0. Hence there are constants such that 



Lin^k + y2 xln^k) 



^i,k 



(13) 



for k = 1, ... ,n. These are determined from the initial conditions. First, 
note that, given the constants Ci^k, we can solve for the infinitesimals Ti^k and 
0^^ and integrate to recover the desired transformation x\ and ti. We will illus- 
trate this calculation with a simple example below. Second, note that we can 
monitor the failure of synchronization of the state estimation Lagrangians by 
simply observing that the left hand side of a Noether invariance relation is not 
a constant. If the left hand side of (13) is not a constant, then we know that the 
current agent Lagrangian is not compatible with the system Lagrangian so that 
Agent i would use our Adapter (sec below) to reset his Estimation Lagrangian. 



6 Simplified Radar Returns Example 

This example is to show how easy the infinitesimal calculations can be. (See [4] 
section 2.2, [26], section 2.1.) A simplified range model of radar returns has the 
following system Lagrangian: 




Now suppose that Agent 1 has infinitesimal transformations 



t\ — t + t(x, t)w 



(14) 



(15) 



xi = X + 6{x, t)w. 



(16) 
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Here w represents the radar returns, which are the sensor outputs. We assume 
that X = x{t) satisfies the Euler-Lagrange equation so that 



d fdL 



Thus 

Then from (11), we get 
d dL 



dt V dx 



dx 



— 2tx + t^x ® t^x^ 



x = 0 2tx 



^ ,, d . n ,t^x^ t^X^ . , 



(17) 



(18) 



(19) 



Expanding the derivative, replacing t^x, and collecting terms, we obtain the 
following relations: 



tx 



,,.6 



-r + t^x^e 



t^x^ dr 

~lh 



= 0 



(20) 



, 2 ^ 

dt 



X® dr 
6 6 9x 



(21) 



tr + r 



dx^ 2 dt 



= 0 



(22) 



^ dr 
2 dx 



(23) 



Note that by (23), r = r(t) is just a function of t. Also (23) and (21) imply that 
^ = 0 so that 6 — 9{x) is just a function of x. This means that (19) and (22) 
are just ordinary differential equations. Solve, getting: 

T — at and 0 = 0^x (24) 

where a is an arbitrary constant. Hence 

ti = t + atw and xi = x 0 2 ^^ 



The constant a is determined by initial conditions. 
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7 Inverse Lagrangians 

The methodology behind MAHCA replaces differential descriptions of evolution 
of state by corresponding descriptions of evolution of state along geodesics of a 
variational problem. This is instantiated in MAHEA in that differential mod- 
els of agent sensors are replaced by Lagrangian cost function models whose 
geodesics generate the same state estimates. Such cost functions carry infor- 
mation not contained in their the Euler-Lagrange equations about connections 
(gauge potentials) and curvature (field strength), as in gauge field theory. (For an 
introduction to these topics, see[8], Ch. 10. For significant physical applications, 
see [6], pp. 79-173.) Different inverse Lagrangians may have different properties 
when used for coordination and synchronization as well, and we too wish to 
make heavy use of associated connections and curvature [19]. 

Here is a brief outline of one inverse method. Interpret the differential e- 
quation as a description of a sensor as an input-output device. Suppose that 
X = g{t,x,u(t)) describes a sensor which at time t with input readingu(t) out- 
puts measurement x{t). 

Define f{t,x) — g{t,x,u{t)), so that the equation is now x — f{t,x). Dif- 
ferentiate with respect to time and now write the result as x = F{t,x,x). In 
the second order Euler-Lagrange equations for a to-be-determined Lagrangian 
L{t,x,x), replace x by F{t,x,x). Differentiate the resulting equation with re- 
spect to t, and get a vector differential equation 

d 

~Tidjxx L±x F L±±F X ® L±x ~ 0 
dt 

with transpose 

~^Lxx + Lxx F F^ L-xx ® Lxx — 0 ( 2 '^) 

Add these two equations, divide the result by two, and get a Lyapunov e- 
quation for F = Lxx- 

= (28) 

Solve this equation for F — Lx± - Then integrate along the integral curves of the 
original differential equation to compute L. 

Remark. If we follow Weierstrass and add time as a state coordinate ([29], 
Ch 2, sec 7) so that y — (xi, . . . , x„, t) and L*(j/, y) — L{t, x, x), we get what is 
called a homogeneous Lagrangian: L*(j/, y),that is one such that: 

L*{y,\y)^XL*{y,y) (29) 

for all positive A, for which the extremals are essentially the same as those of L. 
Differentiation of this equation with respect to A gives Euler’s theorem: 



(26) 



L*{y,y) = SL*^{y,y)y^ 



(30) 
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([28], p. 4, 1.11). A little more computation shows that {L*)^{y,y) — ^y'^ {L‘^)±±x, 
([28], p.6, 1.19b). This eliminates some integration in hnding an L* and therefore 
an L. This is the origin of our Finslcr space approach to hybrid systems ([19]). 



8 The Multiple Agent Hybrid Estimation Architecture 

In this section, we describe the main operational and functional characteristics 
an agent in a MAHEA network. We take for granted the reader has read an 
account of the MAHCA (Multiple Agent Hybrid System architecture) such as 
[18], [12]. Unexplained notions arc from these papers. The MAHEA Architecture 
is implemented as a distributed system composed of agents and a communica- 
tion network which we call the logic communication network. The architecture 
realizing this system operates as an on-line distributed theorem prover. Our ar- 
chitecture interacts with the plant at a series of update times Z\i < Z\2 < . . . . 
These update times are a function of the application. The next update time is 
determined by the interaction between the plant and the agents. At an update 
time, each active agent will receive information from a certain suite of sensors 
and generate estimation actions as a side effect of proving an existentially quan- 
tified lemma which encodes the model of the plant as viewed by the agent. The 
conjunction of lemmas at each instant of time, encodes the desired behavior of 
the entire network. The number of agents in the network is variable. That is, 
the system can spawn new agents and deactivate agents as a function of sys- 
tem demand. Each agent of MAHEA, as a specialization of MAHCA, consists 
of five modules: a Planner, a Dynamic Knowledge Base, a Deductive Inferencer, 
an Adapter and a Knowledge Decoder. See Figure 4. We briefly review the func- 
tionality of an agent in terms of its modules. 

The basic architecture of an estimation agent consists of five modules with 
the following functionality: 

1. Planner. The Planner constructs and repairs the agent state estimation op- 
timization criteria which we refer to as the Estimation Lagrangian associated 
with the agent. In particular, the Planner generates a statement represent- 
ing the desired model of the estimation system as an existentially quantified 
logic expression herein referred to as the Estimation Statement. 

2. Inferencer The Inferencer determines whether there is a state estimate for 
the agent’s relaxed variational state estimation problem which is a near op- 
timal solution where the agent’s Estimation Lagrangian is used as a cost 
function. If there is such a solution, the agent infers a near optimal estima- 
tion and sends data to the other agents. Otherwise it infers failure terms 
and a new state for the agent and reports the failure to the other agents. In 
particular, the Inferencer determines whether the Estimation Statement is 
a theorem in the theory currently active in the knowledge base. If the Esti- 
mation Statement logically follows from the current status of the Knowledge 
Base, the inferencer generates, as a side effect of proving this Estimation S- 
tatement to be true, the current state estimate of the plant. If the Estimation 
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Fig. 4. MAHEA agent architecture 



Statement does not logically follow from the current status of the Knowledge 
Base, that is, the desired behavior is not realizable, the inferencer transmits 
the failed terms to the Adapter module for replacement or modification. The 
term ‘mear optimal” is used deliberately, since, as in the MAHCA architec- 
ture, relaxed solutions to the variational problems are used, which cannot 
generally be physically realized, but arbitrarily close approximations to them 
can be computed and realized. 

3. Adapter The Adapter repairs failure terms and constructs correction terms. 

4. Knowledge Base The Knowledge Base stores and updates the agent’s plant 
model and constraints. The Knowledge Base also stores the requirements of 
operations or processes within the scope of the agent’s estimation problem. 
It also encodes system constraints, inter-agent protocols and constraints, 
sensory data, operational and logic principles and a set of primitive inference 
operations defined in the domain of equational terms. 

5. Knowledge Decoder The Knowledge Decoder receives and translates the 
other agent’s data. 

To better understand how these five modules of the MAHCA architecture 
function in the MAHEA architecture, we outline the basic elements of an agent’s 
model and how it behaves. Let Ai, i = 1, . . . , N{t) denote the agents active at 
the current time t. In our model, t takes values on the real line K. At each time t, 
the status of each agent in the network is given by a point in a smooth manifold 
M . The Estimation Lagrangian L, of an active agent Ai is given by a continuous 
function 



+ 
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where is the positive real line. A point p in the manifold M is represented 
by a data structure of the form: 

p{id, proc{proc-data) , est{estimatiori-data) , in(synch-data) , mp{mult, data)) 



Here id is an identifier taking values in a finite set ID, procQ is a relation char- 
acterizing the status of plant processes, whieh depends on a list of parameters 
labeled proc-data which define the operational, load, and timing characteristics 
of the process involved. The relation est{) captures attributes of the plant be- 
ing represented which depends on a list of parameters labeled estimation-data 
which characterize, among other things, various constraints of the plant repre- 
sentation of an agent at a level of abstraction compatible with the logic commu- 
nication network. The relation m() carries synchronization information of the 
logic communication network. This includes information such as priority level, 
connectivity and time constants. Finally, the relation mp{) carries multiplicity 
information, that is, it represents the level of network usability at this point. 
The associated parameter list, mult-data, is composed of statistical parameters 
reflecting the logic network’s load. 

From an agent’s point of view, the dynamics of the plant is characterized by 
certain trajectories on the manifold M. These trajectories represent the agent 
estimate of the state of the plant plus the flow of information through the network 
and its status. Specifically, we need to define two items: 

(i) The Estimation Lagrangian functions : 

{L^{p,t) : i e I{t)} 

where I{t) is the set of active agents at time t and 

(ii) the actions or estimates issued by the agents. 

These actions are implemented as derivations on the algebra of scalars of the 
manifold. The general structure of an Estimation Lagrangian function for an 
active agent i at time t is given by: 

Li{'P,t) = F,{U^,L,ai){p,t) 

where is a smooth function, L is the vector of Estimation Lagrangian func- 
tions, Ui is the state estimation error function, and is the command action 
issued by the 1-th agent. 

In our application, each command issued by the MAHEA agent is imple- 
mented as a vector field on the manifold M. Each agent constructs its command 
field as a combination of “primitive” predefined vector fields. An integral curve 
associated with a vector field v, denoted by F{t,p) is termed the flow generated 
by V if it satisfies the following conditions: 

semigroup: I'{t,'I'{T,p)) =F{t + T,p) 

initial condition: \P{0,p)=p 

flow: f^F{t,p) =v 
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Let us customize these MAHCA notions for the MAHEA model. An agent 
Ai is active. Let Z\ > 0 be the width of the current decision interval, [t, t + A). 
Let Ui{p, t) be the state estimation error at the beginning of the interval. Agent 
Ai has a set of primitive actions: 

{ I j = ■ ■ ■ i^i, where G TMp for each p G M } 

where TMp is the tangent manifold at p, and as is usual, in geometric control, 
control actions and vector fields are identified. 

During the interval [t,t + A), agent Ai schedules one or more of these ac- 
tions to produce a flow which will reduce the state estimation error. As in 
MAHCA, the schedules are determined by computing an approximate solu- 
tion to a relaxed variational problem. In particular, Ai determines the frac- 
tion aij{p,t) of A that action Vi,j must be executed as a function of the 
current estimation requests Sr,i{t,p) and the vector of estimation Lagrangians 
L{p, t) — (Li(p, t), . . . , t)) of the active agents in the MAHEA network. 

We can express the change in the state estimation error Ui due to the flow over 
the interval A in terms of Vi\p. The evolution of the state estimation error Ui 
over the interval starting at point p and ending at a point p” is given by 

U,{t + A,p'') = U,{t,T,{t + A,p)) (31) 

At time t and at point p E M the estimation error function of agent i is given 
by: 

Ut{p,t) = Ui{p,r) + Sr,r{p,t) T^^Qr,kLk{pA^) (32) 

k 

where t~ is the end point of the previous update interval, Sr,i is the estimation 
request function to agent i, and Qi^k is a multiplier determining the required 
degree of accuracy and the urgency of the estimate of the Agent k that Agent 
i requires. This allocation is determined from the characteristics of the process 
both agents arc estimating and from the process description encoded in the 
agent’s knowledge base. The actual request from agent k to agent i is thus the 
term, Qi,kLk{p,t )- The information sent to agent i by agent k is the state 
estimation function Lk{p,t ) ths end of the previous interval. Finally the 
point p G M carries the current estimate of the process monitored by the agents. 
Agent k thus contribute to Agent f’s new estimate only if Qi k ^ 0. 

This concludes our description of the model. The strategy for activation and 
deactivation of agents is omitted. 



9 Last Remarks 

We believe that the MAHEA agent network is an efficient mechanism for s- 
tate estimation from sensor readings which is extensible, robust, scalable, allows 
cross-checking, and support heterogeneous sensing of heterogeneous processes. 
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Deployment of an agent-based system is very simple. As soon as a new source 
of information is available, a new estimation agent is spawned whose Knowledge 
Base is a model of the plant covered by that source (extensibility). Moreover 
no common representation of the data is required, so the system supports het- 
erogeneous information sources. Thus our architecture allows us to incorporate 
existing models and estimation techniques. The built-in invariance (conserva- 
tion) conditions provide tests for the validity of the data (cross-checking). 

The existing theory of MAHCA (multiple agent hybrid control architecture), 
based on approximations of relaxed variational calculus geodesics (which is all 
that is available in non-convex problems) is adapted to the state estimation 
problem for the plant by constructing a Lagrangian which becomes 0 at points 
which correspond to consistent estimates of the plant and is positive at points 
which are not consistent. This property ensures that when an agent reaches 
consistent estimates of the plant, the evolution of the system produced by the 
flow of the corresponding geodesic field is adapted to the current information of 
the plant as viewed by each agent. 



Process 



Agent 
state spaee 



Product of 
state spaces 
of other agents 





Synchronization 



Fig. 5. The companion agent 



A key result for agent synchronization is the analog of the Thevenin Theorem 
which states in a network with many agents, an individual agent A can view the 
rest of the sensor agents as a single aggregated estimation agent C{A) called 
A’s companion agent. See Figure 5. The fact that an agent sees the rest of the 
estimate agent network as a single equivalent estimation agent is the basis for 
easy scaling up to more agents. 







140 



W. Kohn, A. Nerode, and J.B. Remmel 



The robustness of the sensor agent’s estimates follows from the continuity of 
the relaxed variational geodesic in the parameters of the problem. 

The techniques mentioned in this paper for synchronization and consistency 
of state estimates solve many synchronization problems for our Multiple Agent 
Hybrid Control Architecture. A full exposition will be given in a book in prepa- 
ration. 

Acknowledgments. We thank Prof. John Hubbard for valuable remarks; 
Jennifer Davoren for work on the manuscript, and Xi Krump for the illustrations. 
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Abstract. In this paper, a class of timed Petri nets named programma- 
ble timed Petri nets is used for supervisory control of hybrid systems. In 
particular, the transfer of the continuous state to a region of the state 
space under safety specifications on the discrete and continuous dynam- 
ics is addressed. The switching policy is embedded in the dynamics of 
the underlying Petri net structure and the supervisors are described by 
Petri nets. The discrete specifications are expressed in terms of linear 
constraints on the marking vector and are satisfied by applying supervi- 
sory control of Petri nets based on place invariants. The hybrid system 
switches from a subsystem to another, in a way that the state gradual- 
ly progresses from one equilibrium to another towards the desired tar- 
get equilibrium. The supervisory control algorithm is designed to allow 
switchings to occur only on the intersection of the invariant manifolds. 
Finally, in the case when the continuous dynamics are described by first 
order integrators, the design algorithm is formulated as a linear program- 
ming problem. 



1 Introduction 

In hybrid systems the behavior of interest is governed by interacting continuous 
and discrete dynamic processes. Hybrid control systems typically arise from the 
interaction of discrete planning algorithms and continuous processes, and their 
study is essential in designing discrete event supervisory controllers for con- 
tinuous systems, and central in designing intelligent control systems with a 
high degree of autonomy. The investigation of hybrid systems is creating a 
new and fascinating discipline bridging control engineering, mathematics and 
computer science; further information on hybrid systems may be found in refer- 
ences [1,2, 3, 4, 5, 6]; see also the survey paper [7]. 
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This paper considers systems that arise when computers are used to super- 
vise or synchronize the actions of subsystems described by continuous dynamics 
(that involve continuous variables). Examples of such systems arise in chemical 
process control, command and control networks, power distribution networks, as 
well as distributed manufacturing systems. The size and complexity of such sys- 
tems often requires that the system use a number of distinct operational modes. 
Consequently, these systems can be viewed as supervised systems, in which a 
high-level discrete (-event) supervisor is used to coordinate the actions of vari- 
ous subsystems so that overall system safety is guaranteed. By safety, wo mean 
that pre-specified limits or tolerances on the subsystem states are not violated. 
In the paper, the sets of safe states are characterized by Lyapunov functionals 
and are related to stability properties of the subsystems. So, these systems can 
be viewed as a hybrid mixture of systems with continuous dynamics (contin- 
uous variables) supervised by a switching law generated by a (discrete-event) 
supervisor described by discrete dynamics (discrete variables). 

Petri nets have been used extensively as a tool for modeling, analysis and 
synthesis for discrete event systems. For DES control, Petri nets modeling for- 
malism offers some advantages over finite automata, and it is also useful for 
hybrid systems control. Peleties and DeCarlo [8] presented a model based on 
the work in [9] on the periodicity of symbolic observations of piecewise smooth 
discrete-time systems. This hybrid model is suitable for Petri net based symbolic 
analysis of hybrid systems; the continuous plant is approximated by a Petri net 
and a supervisor consisting of two communicating Petri nets controls the behav- 
ior of the open plant. Lunze et al. [10] proposed a model where Petri nets are 
used as a discrete event representation of the continuous variable system; the 
system and the interface are represented by a Petri net and the supervisor rep- 
resents a mapping of the output event sequence into the input event sequence. 
Several other approaches to modeling of hybrid systems that use Petri nets have 
also been reported in the literature [11,12,13,14,15,16]. 

In this paper, a class of timed Petri nets named programmable timed Petri 
nets [17] is used to model hybrid systems. In particular, it is assumed that the 
switching policy is embedded in an underlying Petri net structure and that the 
supervisors are described also by Petri nets. Petri nets are used instead of finite 
automata because of the following two reasons. The first is the expressiveness 
of Petri nets. Petri net languages include regular languages described by finite 
automata and further, they can model switching policies that describe conflict, 
concurrency, synchronization, and buffer sizes. The second reason is that recent 
results in the supervisory control of discrete-event systems using ordinary Petri 
nets [18] have made possible to design supervisors in an efficient and transparent 
manner; and this methodology is used in this paper. 

In the nonlinear control literature, switching has been used to expand the 
domain of attraction of a control system [19,20]. Here, it is assumed that the 
continuous subsystems admit a family of equilibria and each equilibrium has 
a domain of attraction associated with it. The hybrid system switches from a 
subsystem to another, in a way that the state gradually progresses from one 
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equilibrium to another towards the desired target equilibrium. For the hybrid 
systems of interest in this paper, this idea can be formalized using an invariant 
based approach to the design of hybrid systems [21,22]. This approach introduces 
the notion of a common flow region , which is defined as the set of states whieh 
can be driven to the target region with the same control policy, and gives suffi- 
cient conditions for a set of invariant manifolds to bound common flow regions. 
In this paper, such invariant manifolds are determined by appropriate Lyapunov 
functions. The switchings are allowed to occur only if the continuous state lies 
on the intersection of those invariant manifolds. Since the switching logic is de- 
scribed by a Petri net, only sequences of invariant manifolds that satisfy the 
discrete specifications have to be considered. 

The paper is organized as follows. Section 2 presents programmable timed 
Petri nets which are used in section 3 to model hybrid control systems. In Section 
4 we discuss in detail a Petri net approach to hybrid control which emphasizes 
supervisory control of hybrid systems and we give a simple illustrative example. 
Note that related work has appeared in [23,24]. 



2 Programmable Timed Petri Nets 

Programmable timed Petri nets were introduced in [17] and are used to generate 
the switching logic of the hybrid system. In particular, a programmable timed 
Petri net (PTPN) is a timed Petri net whose places, transitions, and arcs are 
all labeled with formulae representing constraints and reset conditions on the 
rates and times generated by a set of continuous-time systems called clocks. The 
model can seen as an extension of the Alur-Dill hybrid automaton model [25,26] . 

An ordinary Petri net structure [27,28,29] is the 4-tuple Af — {P,T,I,0) 
where P is a finite set of places, T is a finite set of transitions, I ^ P -^T is a 
set of input arcs (from places to transitions), and O -^P is a set of output 
arcs (from transitions to places). The preset and postset of a place p are defines 
''oy ^ — {t \ (t,p)} e O and {t \ (p,t) S /}. The preset and postset of a 

transition t are defined similarly as = {p ] (p, t) € 1} and t—^ {p \ {t,p) G O}. 

The marking of a Petri net is a mapping p : P ^ Z+ from the set of 
places onto the nonnegative integers which assigns to each place p a number of 
tokens p(p). The marking can be represented also by an np-dimensional vector 
p- = (pi, . . . , /ip), where rtp = |P|. The vector /i gives for each place pi, the 
number of tokens in that place, p* = p(pi). To avoid confusion, the marking 
p is interpreted as a mapping when it is appeared with an argument and as a 
vector of nonnegative integers otherwise. The dynamics of ordinary Petri nets 
arc characterized by the evolution of the marking vector which is referred to as 
the state of the net. 

The transition t is enabled when each one of its input places is marked with 
at least one token, p(p) > 0 for all p G An enabled transition may fire. The 
transition t fires by removing one token from each one of its input places and 
by placing one token to each one of its output places. If p(p) and p'(p) denote 
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the marking of place p before and after the firing of enabled transition t, then 

( pip) + 1 if p e f ^ 

p'ip) = I p{p) <S)i a p e (1) 

p[p) otherwise 

The firing of the transition t is described by the firing function q : T {0, 1} 
such that q{t) = 1 if t is firing and q{t) = 0 otherwise. In imtimed Petri nets one 
can prohibit controlled transitions from firing, but cannot force the firing of a 
transition at a particular instant. In a timed Petri net controlled transitions are 
forced to fire, as this can be accomplished by considering the firing functions to be 
functions of a global time. For the timed Petri net, the firing of a transition occurs 
over a time interval [7o,Ty]. The length of this interval is called the transition’s 
holding time. A transition t which starts to fire at time tq is said to be committed. 
During the time that the transition is committed, the network’s marking vector 
is not changed. It is only when the firing is completed at time t/ that the marking 
vector is changed according to equation (1) given above. 

The holding times can be seen as control variables. They can be controlled 
by specifying conditions which cause transitions to fire. The conditions that 
characterize the holding times are represented by logical propositions defined 
over a set of vector dynamical equations, which can be seen as a set of local 
clocks. 

Consider the set, X, of N local clocks where the ith clock Aj is denoted by the 
triple {xi, Xio, no). Xio E 3?" is a real vector representing the clock’s offset, no is 
an initial time (measured with respect to the global clock) indicating when the 
local clock was started, : 3?" ^ 3?" is a Lipschitz continuous automorphism 
over 3?” characterizing the local clock’s rate. It is assumed that the clock rate 
Xi is denoted by the automorphism /. The local time generated by the ith clock 
will be denoted as Xi which is a continuous function over 3?" that is the solution 
to the following initial value problem for t > no j 

dxj „ , , 

Xiino) = x^o■ ( 3 ) 

The state of the «th timer is the ordered pair Zi{T) = (xj(r), i, (<)). The ensemble 
of all the local clock states will denoted by z(t). 

The interval [tq , tj] over which a transition t will be firing is characterized 
by conditions on the “local time” Xi(r) and the “clock rate” Xi{t) of the ith 
timer. These conditions are described by formulae in a propositional logic whose 
atomic formulas are equations over the local times or clock rates of X . In the 
next section, the local clocks are used to describe the continuous dynamics of 
the hybrid systems at each operational mode. The hybrid system switches modes 
based on constraints on the continuous states. The atomic and the propositional 
formulas which will be used to describe the conditions on the states of the vector 
dynamical equations are defined next. Their form is general enough to describe 
a variety of constraints which will be used to characterize the evolution of the 
hybrid system in the next section. 
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Definition 1. An atomic formula, p, takes one of the following forms; 

1. It can be a time constraint of the form h{xi) = 0 or h{xi) < 0 where h : 

^ 3i is a real valued function. This formula means that the clock time Xi 
satisfies the equation. 

2. The atomic formula p can be a rate constraint of the form Xi — f which 
means that the ith clock’s rate i, is equal to the vector field f : 3?" ^ 3?". 

3. Finally, p can be a reset equation of the form Xi(r) = xq which says that 
the ith clock’s local time at global time r is set to the vector xq. 



Definition 2. A well- formed formula or WFF is defined as any expression gen- 
erated by a finite number of application of the following rules; 

1. Any atomic formula is a WFF, 

2. If p and q are WFFs, then p Aq is a WFF. 

3. If p is a WFF, then~p is a WFF. 

The set of all WFFs formed in this manner will be denoted as P. Con- 
sider an ordinary Petri net, Af — {P, T, I, O) and a set of logical timers, X . 
A programmable timed Petri net (PTPN) is denoted by the ordered tuple, 
{Af, X where the functions £p ■. P ^ P, Ip : T ^ P, : I ^ P, 
and £o ■ O ^ P label the places, transition, input arcs, and output arcs (respec- 
tively) of the Petri net Af with WFFs in P. 

The syntax for well formed formulas is defined with respect to the underlying 
Petri net structure of the form Af — (P,T,I,0) and the set of local clocks X. 
The local clock state 2 at time r is said to satisfy a formula p G P if p is “true” 
for the given clock state, z{t). The truth of the well-formed formed formulas is 
understood in the usual sense. 



3 Hybrid Control Systems 

In this section, programmable timed Petri nets arc used to model hybrid systems. 
The hybrid control systems of interest in this paper are described by the following 
equations 



X = fi(t){x{t)) (4) 

i{t) = q{x{t),i{t-)) (5) 

where x{t) : 3? ^ 3?" and i : 3? ^ Z+ denote the continuous and discrete states 
of the system, respectively. The continuous dynamics arc controlled by a finite 
collection of N control strategies 

P = {.fl,.f2,^N} ( 6 ) 

where fi : 3?" ^3?*” ^ 3?" for i G {1, . . . ,N} are locally Lipschitz continuous 
functions. The discrete state of the system is controlled by a successor function 
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q : 3?” — >-Z+ — > Z+ which determines the next possible discrete state i{t) at time 
t given the current continuous state and the “previous” discrete state 

Wc introduce now some additional notation that will be useful in formulating 
the control algorithms later in the section. The firing times of transition t are 
described by cr^{n), n 6 Z+, where cr^{k) C 3? represents the duration of the 
firing of transition t. During the time interval the tokens of the input 

places of transition t do not change. These tokens are put into the output places 
of t upon the completion of firing of the transition, according to the enabling 
condition of the untimed Petri net. We assume that at each time instant exactly 
one transition is firing. In addition, wc assume that 0 < Z\ < oo, for some 

Z\ £ 3?, for all firings n and transitions t. We may easily incorporate in our model 
instantaneous transitions, but these correspond to jumps in the continuous state 
and will not be considered here. The assumption 0 < Z\ eliminates the 

possibility of infinitely many switchings in a finite time interval. 

The control algorithm for the mode selection problem will be based on struc- 
tural information associated with the places of the Petri net. Routing policies for 
timed Petri nets are used usually for resolution of conflicts and were introduced 
in [30] . In our case, we define a mapping v'p (n) : Z+ ^ T for each place p € P, 
where v^{k) identifies the particular transition t G p— ?to which the k^^ token to 
enter place p is to be routed. Note that more than one transition is enabled but 
only one is allowed to actually fire. If the k}^ token is routed to f £ p-^then the 
transition t wins the token, which after a firing time of cr*(fc) is routed to t— ^the 
output places of the transition. 

Next, a firing event is defined as the pair [t, t) which denotes that the tran- 
sition t starts firing at time r. Consider the sequence of firing events 

s = (fio>T-o),(fp,Ti),... , £ {1, . .., A^}, for j = 0, 1,2, . . . (7) 

where j denotes the ordering of the transitions that fire. For example s — 
(fi,To), (tsjTi), . . . denotes that t\ fires at tq, next fires at t\ and so on. The 
firing time intervals are defined by the equation 

= Tfe+l ® Tfc (8) 

At the firing of the network, the transition ti starts firing (at time t^) for 
a^'{k) time units (until Tfc+i). The continuous state of the system during this 
interval evolves according to 

for Tfc -^T < Tfe+i. (9) 

The event projection and the timed projection of the sequence s are defined 
as 



( 10 ) 
( 11 ) 

These arc used later in this section. 



7Ti(s) = io, h, 12 , • • ■ 

7T2(s) = CT*’0 (ko), (ki), . . . 
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4 Supervisor 

The supervisor has two main tasks. The first task is to allow only sequences of 
events that satisfy spccihcations imposed on the discrete-event part of the hy- 
brid plant. In particular, consider the net M of the hybrid system. The objective 
here is to restrict the possible mode switches of the systems to satisfy addition- 
al logical constraints (for example mutual exclusion constraints) that have not 
been taken into consideration in the modeling phase of the hybrid plant. This 
can be accomplished without any information about the continuous dynamic- 
s. The differential equations of the continuous subsystems associated with the 
transitions are used to label these transitions. The second task of the supervisor 
is to enforce firing times that satisfy specifications on the continuous state of the 
plant. In untimcd Petri nets one can prohibit transitions from hring, but cannot 
force the firing of a transition at a particular instant. In a timed Petri net con- 
trolled transitions are forced to fire, as this can be accomplished by considering 
the firing vectors to be functions of the global time r. We will show that for a 
special class of problems, we can hrst determine the routing policy and then the 
firing times that will not violate certain conditions imposed on the continuous 
dynamics. These conditions will be expressed as well formed formulas labeling 
the input and output arcs of the Petri net. 

4.1 Supervisor Control of Petri Nets Based on Place Invariants 

The first step is to satisfy the discrete specifications of the hybrid plant by ap- 
plying DES control methods. We assume that the discrete specifications are de- 
scribed by linear inequalities on the marking vector of the Petri net. A methodol- 
ogy for DES control based on Petri net place invariants has been proposed in [18]. 
A feedback controller based on place invariants is implemented by adding con- 
trol places and arcs to existing transitions in the Petri net structure. Although 
the method was developed for ordinary Petri nets, the introduction of time de- 
lays associated to each transition will not affect the controlled behavior of the 
Petri net with respect to the discrete specifications. The supervisor is used to 
enforce a set of linear constraints on the discrete state of the hybrid plant. These 
constraints can describe a broad variety of problems including forbidden state 
problems, mutual exclusion problems, a class of logical predicates on plant be- 
havior [31], conditions involving the concurrence of events, and the modeling of 
shared resources. 

The system to be controlled is the untimed Petri net A/" = {P, T, I, O), which 
is called the plant net. We assume that the plant net has n places and m tran- 
sitions and its incidence matrix is Dp. The controller net is a Petri net with 
incidence matrix made up of the transitions of the plant net and a separate 
set of places. The controlled net is the Petri net with incidence matrix D made 
up of both the plant and the controller net. The control objective is to enforce 
the discrete state to satisfy constraints of the form 



LyUp 



(12) 
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where fip is the marking vector of the plant net, L is an ric -^n integer matrix, 
b is an ric integer vector, and ric is the number of 1-dimensional constraints 
of the type kl-i'i ^/3- 

This inequality constraint can be transformed to an equality by introducing 
an external Petri net controller that contains places representing nonnegative 
slack variables. Then 

Lfjjp T fJjc — ^ 

where Hc is an ric integer vector which represents the marking of the controller 
places. The structure of the controller net will be computed by observing that the 
introduction of the slack variables forces a set of place invariants on the controlled 
system. A place invariant is defined by an integer vector x that satisfies 

= x^/7.0 (14) 

where /io is the initial marking and fj, any reachable subsequent marking. The 
place invariants of a net are elements of the kernel of the net’s incidence matrix, 
and they can be computed by finding integer solutions to 

x'^D = 0 (15) 



where D is Siii n —>m incidence matrix. The matrix Dc contains the arcs that 
connect the controller places to the transitions of the plant net. The incidence 
matrix D of the closed loop system is given by 



D = 



Dp 

Dc 



(16) 



and the marking vector pL and the initial marking p,o are given by 



M = 






Mo 



Mpo 

Mco 



(17) 



Note that equation (13) is in the form of (14), thus the invariants defined by 
equation (13) on the system (16), (17) must satisfy equation (15). 



\LJ] 



Dp 

Dc 



= 0 



LDp + = 0 



(18) 

(19) 



If Dc is chosen as the solution of equation (19), then the rows of [T, /] are 
elements of the kernel of the net’s incidence matrix. Therefore, they represent 
place invariants of the closed loop systems and equation (13) is satisfied. Since 
Ijl{p) > 0 for all p G P, inequality (12) holds componentwise. The above analysis 
leads to the following proposition presented in [32]. 



Proposition 1. The Petri net controller with incidence matrix Dc and initial 
marking /Xcq, which enforces the constraints Lp,p -^b when included in the closed 
loop system (16) with marking (17) is defined by 



Dc = ®LDp 



( 20 ) 
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with initial m,arking 

Mco — ^0 ^t^po 

assuming that the transitions with arcs from are controllable, observable, and 
that /ico ^0- 

This proposition designs a controller that enforces the linear constraints 
L/i b under the assumption that the controller will enable or inhibit only 
controllable and observable transitions. These results have been extended for 
handling uncontrollable and unobservable transitions in [33] . In the hybrid sys- 
tems case, we have associated transitions to continuous subsystems described by 
differential equations. It is assumed that the supervisor can force and observe 
the firing of the transitions. This is accomplished by imposing conditions de- 
scribed by well-formed formulas on the input and output arcs of the transitions, 
as described in the next section. 

4.2 Hybrid Strategy based on Equilibria 

In the nonlinear control literature, switching has been used to expand the domain 
of attraction of control systems [19,20]. In the hybrid systems case, we assume 
that the continuous part admits a family of equilibria corresponding to different 
symbolic inputs generated by the discrete event part. Each equilibrium has a 
domain of attraction associated with it. The idea is to switch at discrete time 
instants from one symbolic input to another in a way that the system gradually 
progresses from one equilibrium to another towards the final equilibrium. 

This idea can be formalized using an invariant based approach for hybrid 
systems proposed in [21,22]. A common flow region for a given target region, is 
defined as a set of states which can be driven to the target region with the same 
control policy. The approach as described by Stiver et al. considers common flow 
regions which are bounded by invariant hypersurfaces, cap boundaries and an 
exit boundary. Invariant hypersurfaces and cap boundaries which are described 
next in the section, form manifolds to bound a region so that the state trajectory 
can leave the region only through the exit boundary. In [21] sufficient conditions 
for a set of hypersurfaces to form a common flow region were established. Here, 
a Lyapunov approach is followed to efficiently compute hypersurfaces that form 
common flow regions for each control policy. Each common flow region is identi- 
fied as a subset of an invariant manifold defined by a Lyapunov functional and 
is associated with a control policy. Since the switching function is generated by 
the underlying Petri net only sequences of invariant manifolds that correspond 
to control policies which satisfy the discrete specifications have to be examined. 

Definition 3. For the continuous part of the hybrid plant, the set B is a com- 
mon flow region for a given region R if 

Vx(to) € B, 3ti,t2, to < ti < t 2 



such that 



x{t) £ B, t -^ti 
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and 

x{t) G i?, ti < t <t 2 

subject to 

x{t) = fi{x{t)) 

In [21] two proposition are given which provide sufficient conditions for a set 
of hypersurfaces to form a common flow region. These hypersurfaces can be either 
invariant under the vector field of the given control policy or cap boundaries for 
the given vector field. Invariant hypersurfaces and cap boundaries form manifolds 
to bound a common flow region, so that the state trajectories cannot cross those 
manifolds. 

Definition 4. A set M ^ X is said to be invariant with respect to the system 
X = f{x) if x{to) G M ^ x{t) E M, Vt E 3?. 

Consider the set of smooth hypersurfaces, {hi, i E Ib ^2^}. The hypersur- 
faces is a set of smooth functionals {hi \ 3?" — > 3?, i E Ib{, defined on the state 
space of the plant. Each functional must satisfy the condition 

7^ 0, G N{hi) (22) 

which ensures that the null space of the functional A/”(/ii) = {(J G 3?" : hi{f) = 0} 
forms an n ® 1 dimensional manifold separating the state space. 

Invariant hypersurfaces For a hypersurfacc hi to be invariant under the vector 
field / of the given control policy, the following condition must be satisfied 

g/(0 = 0 (23) 

The set of all invariant hypersurfaces can be found in terms of n (8> 1 functionally 
independent mappings which form the basis for the desired set of functionals, 
{hi}. This basis is obtained by solving the characteristic equation 

dx\ dx'2 dxji (oA\ 

fl{x) faix) fn{x) 

where fj{x) is the jth element of f{x) {f{x) is used rather than fi{x) to avoid 
subscript confusion). 

Cap boundaries For a hypersurface he to form a cap boundary for a giv- 
en vector field / and common flow region B, the following condition must be 
satisfied 

^(0 < 0, ve G B n AT(/ie) (25) 

Consider the hypersurface heix) that forms a cap boundary for the common flow 
region B. Assume that there exists an appropriate Lyapunov function V{x) for 
the vector fields / such that 



V{x) > 0, Vx G S 
V{x) ^ oo as ||x|| ^ oc 
V{x) < 0, Vx G I? 



( 26 ) 
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then = {x S 3?"| V(x) c} is bounded and the hypersnrface hc{x) — 
V (x) 0 c is a cap boundary candidate. The constant parameter c can be selected 
appropriately so that the hypersnrface hc{x) which bounds the common flow 
region B satisfies certain safety constraints. 

As it was recognized in [21], the task of determining suitable invariant hy- 
persurfaces is very difficult in general; for special cases (e.g. integrator systems), 
the differential equation (24) was solved analytically, otherwise an algorithm 
which was computationally inefficient was used. Here, we introduce a Lyapunov 
approach to determine cap boundaries. This approach is more efficient and can 
be applied to a larger class of systems; furthermore, the design based on Lya- 
punov functions exhibits desirable robustness properties. However, by assuming 
that the common flow regions are bounded by manifolds defined by Lyapunov 
functionals, wc impose restrictive conditions on the dynamics of the continuous 
subsystems. In most of the cases, these conditions are quite restrictive but they 
suggest a systematic way to compute common flow regions. For example, we re- 
quire stability (in the region of interest) for the continuous subsystems because 
then we can systematically approximate the region of attraction of an equilib- 
rium. The next proposition gives sufficient conditions for the state to progress 
from one equilibrium to another. 

Proposition 2. Let G S satisfy the following assumptions 

1. Each fi is globally Lipschitz and admits an isolated equilibrium point x*, and 
Xi IS asymptotically stable w.r.t. fi. 

2. For each fi there exists an appropriate Lyapunov function V) : 3?" ^ 3? and 
42j = {.X € 3?“| Vi{x) ^Ci} such that 

V(x) > 0 , Vx G f2i 

V{x) —> oc as ||x|| ^ oo (27) 

V{x) < 0, Vx G fii 

In addition, assume that D Ili^ 7 ^ 0 and x,j ^ R' — int{12^^ D 42^2 ), then for 
every xq G there exists a switching sequence 

s{xo,to) = (^l,cr*■l(fcg))^(i2,CT‘‘2(fcl)) 

which drives the state to a region R of the equilibrium point Xj^ . 

Proof. Let 12 = 12*^ \ fii^ and define the hypersurface hc{x) — dI2 n 9f2jj and 
he{x) = BQ n BQi^. Since .x^^ G R! is an asymptotically equilibrium point for 
fii, 12 is a common flow region for R' — int{f2i^ n Let 12' — f2i^ \ R and 
define the hypersurface hc{x) — Bf2i^ and h(.{x) — OR, then f2' is a common 
flow region for the target region R. 

If we assume that the switching logic of the hybrid system is described by 
a DES (here a programmable timed Petri net), only sequences of vector fields 
that correspond to control policies that satisfy the discrete specifications have 
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to be considered. The control policies that satisfy the discrete specifications are 
exactly those that are accepted by the controlled Petri net which includes also 
the supervisor designed using the methodology based on place invariants (see 
above) . They can be determined by identifying the periodic behavior of the Petri 
net. Several methods have been proposed to determine the periodic behavior of 
Petri nets (reachability tree, transition invariants, unfolding). For this problem, 
the size of the Petri net depends on the number of available symbolic inputs, 
hence we can efficiently identify all the periodic behaviors by computing the 
fundamental cycles of the reachability tree. 

The underlying Petri net structure, which generates the switching policy 
offers two important advantages. First, it makes possible to efficiently design the 
supervisor that satisfy speciheations that frequently appear in complex systems 
such as generalized mutual exclusion constraints. Second, it reduces considerably 
the search for common flow regions, since only desirable switching strategies 
generated by the controlled Petri net have to be examined. 

The following corollary gives sufficient conditions for a switching sequence 
generated by the controlled Petri net to drive the continuous state xq to a target 
region of the state space. It is assumed that the initial conditions belong to the 
region of attraction f2ig of the hrst control policy and that it progresses towards 
Xi^ € by allowing switchings to occur on the intersection Pi of 

consecutive invariant manifolds. In the case when all the pairs of control policies 
satisfy Proposition 2, the set f2^. n will be nonempty and the proof is 

clear. 

Corollary 1. Suppose there exists a switching sequence with event projection 
7Ti(s) = io,ii, . . . ,im accepted by the controlled Petri net such that every pair 
satisfies Proposition 1 . Given a target region R such that Xi^ G 
int{R), there exists routing policy zzP(n) to drive the continuous state from any 
initial condition xq G to the region R in finite time. The firing time intervals 
a^{n) will be chosen so that the switchings occur while x G int{Qi. n 

Remark The condition that every pair {fij , satisfies Proposition 2 can be 

relaxed by allowing intermediate transitions which will keep the continuous state 
in the domain of attraction of 

The supervisor is implemented by assigning well-formed formulas to the 
places, transitions, input and output arcs of the controlled Petri net. Let {R}, i — 
1, . . . , n be the set of hypersurfaces that bound a region M of the state space X. 
We can use the following well- formed formulas to describe that x G M. 

£ ^ Pi Ap2 /\ ■ ■ ■ Apn (28) 

where pi is a constraint of the form hfix) < 0. Consider a pair of vector fields 
ifijyfij+i) that satisfy Proposition 2 and let B^.,Ri^ and be the 

corresponding common flow and target regions. From Proposition 2 we have 
that the target region Ri^ coincides with the common flow region The 

switching algorithm is implemented by the following labeling functions, where 
p E P is the place to connect the output arc of ti^ to the input arc of 




154 



X.D. Koutsoukos and P.J. Antsaklis 



1. £p{p) is chosen to be a tautology. 

2. are chosen to be the atomic rate formulas x = fij{x) and 
X = fij^i{x) respectively. 

3. ,p)) is chosen to be a WFF of the form (28) representing that x £ Ri^ . 

4. £/((p, is chosen to be a WFF of the form (28) representing that x £ 

Assuming that transition ti^ is firing, the next transition to fire, is deter- 
mined by the routing policy iy^{n) so that the pair (/j^. , satisfies Proposi- 

tion 2. Transition will fire only when the firing time intervals cr‘(n) assign 
true values to the logic formulas ,p)) and £j{{p,ti.^^)). For the initializa- 

tion of the hybrid system we assume that £i{{p,tig)) is a tautology. 

Remark In the case when the Petri net is live and the event projection generated 
by the controlled Petri net 7ti(s) is an infinite sequence that satisfies Corollary 1, 
the hybrid system exhibits a periodic behavior in the sense that the continuous 
state is visiting periodically neighborhoods of the equilibria. 



Affine Systems A class of systems that satisfy the conditions for superviso- 
ry control design of the previous section is the affine systems. They represent 
physical systems that are described by linear ordinary differential equations with 
one additional assumption. The input is allowed to take a finite number of pre- 
specified constant values. 

Consider the case the continuous dynamics are described by 
x = Ax + Ci, Tk < Tfc+i 

where Ci £ W ^3?" a finite set of control vectors and the matrix A G 3?"^" is 
Hurwitz. 

Let fi{x) = Ax + Ci, then Xi = is a globally asymptotically stable 

equilibrium point for x — fi (x) . In view of the global asymptotic stability of each 
equilibrium point, it is clear that Proposition 2 holds for every pair of control 
inputs. The values for the control input can be selected so that the continuous 
state can be driven to prescribed regions of the state space. 



Example: Hybrid System Describing Resource Contention Consider 
the case of two different processes that use the same resource to carry out their 
operations. This is a conflict situation which stems from the resource contention. 
More specifically, assume that each process consists of two different operations 
which are described by ordinary differential equations and the switching policy 
is represented by the Petri net in Fig. 1. This situation arises frequently in 
physical systems when different processes share the same resources. We will use 
this Petri net to describe the switching policy for two examples that follow. The 
first example is a temperature control system where the continuous dynamics 
are described by afline systems. In the second example, we consider a hybrid 
system with continuous dynamics described by first order integrators. 
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Fig. 1. Petri net describing the switching policy of the hybrid plant. 



The incidence matrix of the plant net is 



Dp 



1 0 0 

1 0 0 

0 0 (g)l 1 

0 0 1 ( 8)1 



(29) 



and the initial condition of the marking vector — [1,0, 1,0]^ . We consider 
the mutual exclusion constraint L/ip D b. where L = [0, 1, 0, 1] and 6=1. Using 
Proposition 1 the closed loop system has the incidence matrix 



D = 



81 1 0 0 
1 81 0 0 
0 0 81 1 
0 0 1 81 
81 1 81 1 



(30) 



and initial condition /iq — [1,0, 1,0, 1]. The last row of the incidence matrix D 
represents the Petri net supervisor. 

The controlled Petri net is shown in Fig. 2 and describes the switching policy 
for the hybrid system which satisfies the mutual exclusion constraint. 



Temperature Control System Let a temperature control system be described 
by the electrical circuit shown in hgure 3. Here, an electrical analog of the tem- 
perature control system is used by considering the temperature being analogous 
to electric voltage, heat quantity to current, heat capacity to capacitance, and 
thermal resistance to electrical resistance. The control objective is to control the 
temperature at a point at the system by applying the heat input at a different 
point. The temperature control example is used in [34] to illustrate PID control 
design. Here, we assume that only discrete levels are available for the current 
(heat) input (u). 

Let xi and X 2 denote the voltages across the capacitors Ci and C 2 respec- 
tively. Suppose that the (voltages) temperatures xi and X 2 arc to be controlled 
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Fig. 2. The controlled Petri net of the resource contention example 




Fig. 3. The temperature control system 



by changing the (current) heat input u, which is allowed to take finitely many 
discrete values. Consider that the numerical values of the electrical elements are 

Ri — i?2 = 1 

Cl = Ca = 1 



Then (using Kirchhoff ’s laws) the system is described by the state-space equation 



ii _ <8>1 1 x\ 

±2 1 < 8>2 X2 




(31) 



Assume that qi, i = 1, . . . , N are the available discrete levels for the input that 
correspond to the available control policies. It is easily verified that the matrix 
A is Hurwitz and therefore Xj = where 



A = 



'oi r 

1 ®2j ’ 




is a globally asymptotically stable equilibrium point for the system x = Ax -\- Ci. 

We assume that the discrete levels of the heat input are qi = ®10,(72 = 
10, ga = ®20, and gi = 20 and that the switching policy is described by the Petri 
net in Fig. 2 representing resource contention, where transition ti corresponds 
to the control input Ci. In this case, the supervisor will determine the routing 
policy and the hring time intervals so that the continuous state of the hybrid 
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Fig. 4. Periodic behavior of the hybrid system 



system visits periodically neighborhoods of the equilibria (Fig. 4). We consider a 
ball Bi{xi,r) of radius r centered at Xi, i — 1, 2, 3, 4 and we label the input and 
output arcs of the transitions with the WFFs io{{U,p)) — ^i{{p,U+i)) — x £ 
Bi{xi,r). Then the supervisor allows switchings to occur only when the above 
logical propositions are true. 



5 Supervisory Control Design for First Order Integrators 

Recently, attention has been focused on a particular class of hybrid systems in 
which the continuous dynamics are governed by the differential equation x{t) — 
c, where c G 3?" [35,36,37,38]. In [36] hybrid systems with continuous dynamics 
described by first order integrators are used for the control of batch processes. In 
the case when the continuous dynamics are described by first order integrators, 
the previous algorithm cannot be applied since the continuous part does not 
admit any isolated equilibria. In the following, we present an algorithm based on 
a similar idea as in the case of multiple equilibria. For each symbolic input there 
exist a family of invariant sets. The switching from a symbolic input to another 
occur at discrete time instants in a way that the system gradually progresses 
towards the target region. We determine a sequence of events that drive the 
state to the prescribed target region by solving a linear programming problem 
as in [38]. 
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5.1 Hybrid Plant 

The continuous state of the hybrid plant evolves in X G 3?" and is described by 
first order integrators 

x{t) = Ci, Tk e Ti e Tfc+i 

where G IT G 3?" a finite set of control vectors and T denotes the global time. 

It is assumed as in the section 3 that the switching strategy is embedded in 
a programmable timed Petri net. Each transition is associated with a control 
policy. This assignment is defined by the labeling function lr{t) : T ^ S which 
is chosen to be an atomic rate formula of the form x = Ci. 

Definition 5. A set C is said to be a finitely generated cone if it has the form 



C — lx : X — ^ TjCj, Tj G 0, Cj G 3?", j = 1, . . . , r 
[ j=i 

In the following, we assume that the finitely generated cone by the set W 
coincides with the continuous state space X. This assumption guarantees that 
continuous specifications such as state targeting can be satisfied everywhere in 
the state space X. 

The system x — c admits a family of invariant sets described by the equation 
X = ct + xq, which represents a family of parallel lines parameterized in t. The 
path from the initial state to the target region will be found by solving a linear 
programming problem to determine the time interval the state of the system 
evolves in each particular invariant set. 



Supervisor The procedure for the design of the supervisor is similar to that 
of the previous section. First, applying DES control methods we construct a 
controlled Petri net which satisfies the discrete specifications. Assume now that 
the control objective is to drive the state to the target region T G X. The task of 
the supervisor is to select at time Tfc a control policy accepted by the Petri net, 
and to decide for how long it should be active. A systematic way to determine 
the temporal and routing data is first to identify all the periodic behaviors the 
controlled Petri net can generate and then assign temporal constraints to the 
switching times. 

Proposition 3. Consider the switching sequence 

s(to,xo) = ^0 = to 



such that 

1. The event projection 7ti(s) = io, fi, * 2 , • • • , consists of the transitions 
which form all the fundamental cycles. 
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2. The timed projecfAon tt 2 {s) — cr‘*o (fcg), (t‘‘i . . . , cr**™ satisfies the 

following conditions “ U (ind 0 < A £ a^{n) < oo for all tran- 

sitions and firings; t = [ri,...,TAr] is the solution of the following linear 
programming problem 

T 

min a r 



subject to : 



Xf = xo + X!ili nci e T 

r, e A, i = 1, . . ,,N 



where t is the vector of operation times to be determined, a is a weighting 
vector, Xf is the response of the continuous part at the tirn,e instant when 
the switching sequence s{to,xo) is completed. 



Then, the continuous state x G ft is driven to the target region in finite time 
t = Li=iu- 

Proof. By integrating the state equation (5.1) all the reachable states Xf from 
initial state xq are given by 



N 

X/ = xo + ^TiCi (32) 

^=1 



where Xi represents the total time the corresponding control policy Ci is active. 
Although there is no unique switching policy to satisfy the convex constrain 
Xf £ T, the solutioir of the linear programming problem is unique aird gives the 
control law that drives the state to the target region in minimum time. 



Additionally, the necessary number of switches can be minimized by con- 
sidering one firing of each transition. Additional safety conditions expressed as 
convex constraints can be incorporated in the linear programming problem. The 
supervisor solves the above linear programming problem and labels the places, 
transitions, input and output arcs of the controlled Petri net as follows 

1. £p{p) is cho.sen to be a tautology. 

2. Tr{ti) is chosen to be the atomic rate formulas x = Cj. 

3. £i{{p,ti)) and £o{{ti,p)) are chosen to be time constraints implement by a 
local timer so that Xlfc cr**(fc) = r,. 



Example: First Order Integrators We assume now that the continuous part 
of the hybrid plant consists of a set of first order integrators 



X = c, £ 3?^, 

C = [C1,C2,C3,C4] 



0.5 01 1 01 
1 1 1 00.4 



and we associate the differential equation x — Ci with the transition ti of the 
Petri net in Fig. 1. The control objective is to drive the state from the initial 
condition xg = [1, 01]^ to the convex region T of the state space where 






|x £ : 



1 

1 



£ X £ 



1.1 

1.1 
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Fig. 5. The trajectory of the continuous state. 



According to Proposition 3 we formulate the following linear programming 
problem 



min (ri + T 2 + ts + T 4 ) 



subject to : 



Xf = Xo + TlCl + T2C2 + T3C3 + T4C4 G T 

n e A 



where A = 0.1. The solution of the linear programming problem gives 



n = 0.6585, T 2 = 0.7554; T 3 = 0.6261, T 4 = 0.1 

and we can drive the state from xq to T with one firing of each transition by 
setting = Tj, i = 1, 2, 3, 4. The trajectory of the continuous state is shown in 
Fig. 5. 



6 Conclusions 

In this paper, supervisory control of hybrid systems was addressed using a class 
of timed Petri nets named programmable timed Petri nets. New methodologies 
were introduced and algorithms were derived to address these issues. Sufficien- 
t conditions for supervisory control design were presented. For the case when 
the plant is a collection of affine systems or first order integrators with switch- 
ing logic generated by a programmable timed Petri net, efficient algorithms for 
supervisory control synthesis were developed. 
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Abstract. This case study compares the usefulness and applicability of 
eight computer tools with respect to the validation of logic control pro- 
grams f(5r contimwrus j>rocesses. Six simulation j>ackages (Taykrr’s Mat- 
LAB-based simulator, Simulink/StateFlow, gPROMS, Shift, Dymo- 
LA, and BaSiP) and two verification tools (SMV and HyTech) were 
applied to a single process control example with non-trivial continuous 
dynamics. The paper presents a detailed description of this benchmark 
example. Short introductions to the tools are given and the application 
results are decribed and discussed with emphasis on the suitability to 
the problem and the numerical performance. 



1 Introduction 

This contribution reports on a case study in the application of tool-based meth- 
ods for a systematic analysis of hybrid systems. In particular, the study is fo- 
cussed on continuous processes controlled by logic controllers, as this class of 
hybrid systems is widespread among process industry plants. To achieve compa- 
rable results, all methods and tools under consideration were applied to the same 
example. We chose a simple process from our laboratory for this purpose. It will 
be introduced in detail in the following section. This benchmark example differs 
from other examples in the literature (as, for example, the train-gate-controller 
[LS87] or the steam boiler problem [ABL96]) in the following respects: 

1. The problem is formulated such that simulation tools as well as verification 
tools can be applied meaningfully and will find particular challenges. In the 
case of simulation, the user must identify the worst-cases properly when non- 
determinism comes into play, whereas the tool has to handle event-triggered 
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switching correctly. The main challenge in applying verification tools is to 
find a necessary abstraction of the continuous dynamics. 

2. Although the plant looks quite simple (two connected water tanks, see next 
section) and the underlying physics arc easy to understand, the continuous 
dynamics are far from being trivial (as can be seen by the Eqs. (5) - (8)). 
As a consequence, it is not obvious on first sight how a suitable abstraction 
can be made, if verification tools based on models with simpler dynamics (e. 
g. HyTech with Linear Hybrid Automata) shall be applied. 

3. The example is an existing plant and not a purely idealized model. This 
becomes obvious in the models of the valves’ dynamics which were deter- 
mined experimentally (Eqs. 7 and 8). Consequently, we are ensured that no 
unrealistic simplifications of the problem are introdneed from the outset. 
Also, since the theoretical results can be checked at the real equipment, it is 
possible to investigate the effects of modeling errors or uncertainties. 

4. The problem statement corresponds to an every-day situation in process 
automation engineering: An already developed control program has to be 
checked with respect to a non-formal, natural language requirement specifi- 
cation based on some knowledge about the process to control. Consequently, 
we use a common industrial representation for the control code in the exam- 
ple. 

The paper is organized as follows. In Sec. 2, we describe the two tanks bench- 
mark problem in detail. Section 3 is devoted to the simulation tools which are 
Taylor’s MATLAB-based simulator [TK96], Simulink/StateFlow [Mat97], G- 
PROMS [PB94], Shift [DGS96], Dymola [Elm93] andBASiP [WFSE96]. Each 
tool is introduced shortly and the application results and user experiences are 
described. The numerical performance is compared with respect to the accuracy 
of state event detection and to computational efficiency. In Sec. 4, the suitability 
of the verification tools SMV [CGL94] and HyTech [HHW97] is evaluated. The 
paper ends with a discussion of the results. 

2 The benchmark example 

2.1 The plant 

The system under consideration is a small laboratory plant which is used for an 
practical exercise in continuous control for chemical engineering students at the 
University of Dortmund. It has been used for research purposes in the past, e. g. 
as a benchmark for nonlinear control design methods [HE94] or as an illustration 
of modeling concepts for hybrid systems [HK96,SKHP97]. The plant consists of 
two connected cylindric tanks T\ and T 2 which are situated on different levels 
(see Fig. 1). The diameter of Tf is 12 cm, the one of T 2 is 5 cm. Both tanks are 
1 m in height. The connecting pipe is attached to T 2 at a height of H = 0.39 m. 
The plant is operated with water. The incoming flow V controlled by the valve 
^input- Since it has only two positions, open or closed, the input flow is given 
by: 
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s!/from the controller 




Fig. 1. Scheme of the laboratory plant 






Ol/h, if closed, 

400 l/h, if Vinput open. 



( 1 ) 



The two other valves, V\ and V 2 , are control valves which can take on any 
(normalized) position between 0 (completely open) and 80 (completely closed): 



Fi,F 2 e [0,80], (2) 

Vi and V 2 are controlled by a binary signal: open valve and close valve. 
Opening and closing the valves is relatively slow: The complete process takes 
approximately 80 seconds. It can be assumed that the opening and closing speed 
is constant: 



A|, |A| = l/s — const. (3) 

The dynamics of the two levels hi and /12 can be derived from the mass 
balances for both tanks {Ai and A 2 are the base areas of Ti or T 2 , resp.): 
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ill 



{V,n^Vi2) • (Fi 2 -^out) 

A, = 



( 4 ) 



Applying Torricelli’s law yields the following model for the flows V \2 and 
4^oMt coming out of the two tanks. Note from Eq. 5 that the autonomous system 
already shows hybrid dynamics as the valid model for V 12 has to be switched 
from one mode to the other when /12 crosses the threshold H . 



V 12 



Ki[Pi)- iJhi-{h2-H), -iih2>H, 

Ki{Pi)-y/hl, \ih2<H 



( 5 ) 



^out — K2{P2) ■ y/^. (6) 

The coefficients K\ and K 2 in Eq. 5 and Eq. 6 are necessary to represent 
the dependencies between the flows and the valve positions. The corresponding 
relations were determined experimentally: 



Ki{Pi) 



■ 10 



-4 . g-6-10®®-F{* 






if 0 < Pi < 80, 
if Pi = 80. 



K2{P2) 



2,26- 10-^ 






,5/2 



if 0 < P 2 < 80, 
if P 2 = 80. 



( 7 ) 

( 8 ) 



2.2 The controller 

Fig. 2 and 3 show the control program for the example process. The repre- 
sentation is in accordance with the international standard lEC 1131-3 for Pro- 
grammable Logic Controllers [Int93]. The program is divided into two parts: the 
declaration part (Fig. 2) and the program body (Fig. 3). For the program body 
the language Sequential Function Chart (SFC) is chosen because it is particularly 
suited for sequence control applications. 

The program does the following: After the start command the process is in 
“start-up-phase” consisting of the steps 1 and 2. During step 1, Ti is filled by 
opening Vinput (S stands for set) while Vi is kept closed. After the period Timel 
the transition to step 2 takes place (the variable <stepname>.T can be used 
according to lEC 1131-3 to determine the current duration of an active step). 
When step 2 becomes active, Vi will be opened. After Time2 the “start-up phase” 
is finished and the “stationary operation” begins in which the process is either 
in step 3 or step 4. The difference between these two steps is that in step 3 the 
output valve V 2 is opened whereas in step 4 it is closed (R stands for reset). Step 
3 is active as long as the level in T 2 remains above the lower threshold L_minus. 
As soon as it drops below, the controller moves to step 4 until the level rises 
above the upper limit L_plus. 
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PROGRAM TwoTanks 
VAR_INPUT 

Start : BOOL; (* Input from environment: start process *) 

Level : REAL (0 .. 1000) ; (* Level in tank 2 *) 

END_VAR 

VAR_ OUTPUT 

V_Input : BOOL := FALSE; (* Output: open/close input valve +) 

V_1 : BOOL := FALSE; (* Output: open/close connection valve *) 

V_2 : BOOL := FALSE; (* Output: open/close output valve *) 

(* for all valves: TRUE = open, FALSE = close *) 

END_VAR 

VAR CONSTANT 

Timel : TIME := T#90s; (* duration of step 1 *) 

Time2 : TIME := T#20s; (+ duration of step 2*) 

L_plus: REAL(0 . . 1000) := 940; (♦ upper limit for level tEuik 2 *) 
L_minus: REAL(0 . . 1000) := 160; (* lower limit for level tank 2 *) 
END_VAR 



Fig. 2. Control program for the example: declaration part 




Fig. 3. Control program for the example: program body (SFC) 
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2.3 The requirements 

The desired behavior of the two tanks system is specified by the following two 
requirements: 

1. No overflow may occur. 

2. The process must reach an equilibrium state in finite time. This means, the 
level in T 2 must not swing periodically between L_minus und L_plus in the 
stationary operation mode. Consequently, the control sequence has to stay 
in step 3 eventually. 

2.4 The different cases 

The requirements given in the previous section have to be checked for five differ- 
ent sets of control parameters (see Table 1). These values were chosen to capture 
most of the interesting scenarios which can take place in the process: 1. overflow 
of Ti, 2. overflow of T 2 , 3. equilibrium state, 4. periodic behavior. The fifth set of 
parameters leads very closely to case 2 but does not violate the requirement. It 
is especially suited to determine the necessary grade of computational accuracy 
of the tools. 





TimGl 


TimG2 


L_plus 


L_minus 


1 


T#90s 


T#20s 


940 


160 


2 


T#70s 


T#30s 


940 


160 


3 


T#70s 


T#20s 


940 


160 


4 


T#60s 


T#25s 


900 


300 


5 


T#70s 


T#26.85s 


940 


160 



Table 1. Different sets of control parameters for the program TwoTanks 



In the first step of the analysis, it is assumed that no disturbances arise 
during the plant operation. Obviously, in this case the problem is purely de- 
terministic and can be solved by proper simulation. To provide a problem for 
which the verification tools can employ their particular strengths in handling 
nondeterminism, the following scenario can be considered in a second step: A 
leakage may appear in tank Ti or in the pipe between Ti and Vi. If this happens, 
approximately 10% of V 12 will be lost. It can be assumed that the leakage will 
be discovered and closed within 2 minutes and that it will not appear again for 
the next 20 minutes after repairing. 

3 Application of the simulation tools 

3.1 Taylor’s Matlab-based simulator 

As a part of a project to define a modeling language for a broad class of hybrid 
systems, a hybrid simulation facility for the widely used modeling and simulation 




Tool- Aided Analysis of Discretely Controlled Continuous Systems 



169 



environment Matlab has been developed by Taylor [TK96]. It facilitates the 
detection and handling of state events in continuous time systems and thus 
supports jumps in the system trajectory as well as switchings of the model 
structure. 

The evolution of the continuous state x is characterized by a set of differen- 
tial equations x — f{x, m, t) depending on the mode input m. State events are 
detected by zero-crossings of a flag variable (j) involving the state, time and the 
mode. Each integration point is calculated as a trial point according to the cur- 
rently valid set of differential equations first, and is only added to the trajectory 
if no change of sign is detected. Otherwise, an iterative procedure is initialized 
to find a step-size h such that the flag variable equals zero (within a specified 
accuracy). Before the integration proceeds, the model state may be reset, if nec- 
essary, and the new mode is calculated. Thus, contrary to standard integration 
routines, it is ensured that mode changes will only take place at the end of an 
integration step. 

The rigorous handling of state events requires an extended Matlab function 
block (Fig. 4) to determine, at each integration step and during the iteration pro- 
cedure, not only the derivative of the continuous state but also the flag variable 
f. In addition, if a state reset becomes necessary, a new value r for the continuous 
state variable x is calculated by a supplementary function h. As an additional 
input, the mode value m, influences the calculations. Since it is computed as the 
sign of the flag variable (p, it is restricted to the values 1, -1 or 0. 




X -f(x,m,t) 
(j) = ^x,m,t) 
r - h(x,m,t) 




Fig. 4. Extended Matlab function block 



Modeling the two tanks system was realized by dehning such a function block 
which describes the evolution of the position of the valves and the liquid levels 
depending on the related mode. In case of the valves three different modes have to 
be distinguished, corresponding to opening, stagnating and closing. The related 
flag variables depend on whether a critical level is reached and on the actual 
valve-position. To consider the coupling/decoupling of the tanks’ dynamics one 
more flag variable had to be dehned depending on the level in tank two. The 
related mode only takes on the values 1 or -1. In this example, a reinitialisation 
of the state vector at switching points was not necessary. However, due to the 
way of the mode calculation, the flag variables which are related to the valve 
positions have to be reset to zero whenever a valve is completely closed/opened so 
that the corresponding mode switches to zero (stagnating). Since the integration 
algorithm currently does not support the handling of time events, an additional 
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script file has to be created which defines the sequence of integrations. At the 
beginning of each integration procedure, the flag variables are initialized within 
the function block according to a global variable that represents the current step 
in the control procedure. The system was simulated using an extended variable- 
step Runge-Kutta-Fehlberg integration routine. The simulation results can easily 
be visualized using the routines provided by Matlab as can be seen in Fig. 5 
for the parameter set 1. Note that h\ is not limited to lin in this model because 
we were only interested in determing the overflow of T\ . 

To summarize, the simulator could handle the benchmark problem correctly. 
The accurate handling of state events provided correct results for every set of 
parameters. The described restriction due to the calculation of the mode as the 
sign of a flag variable did not apply in this case because for each mode only two 
or three different values had to be distinguished. However, it complicates the 
definition of the flag variables and requires a comprehension of the underlying 
integration algorithm to ensure correct iteration results. 




Fig. 5. Visualization of simulation results by Matlab (parameter set 1) 



3.2 Simulink/StateFlow 

Another possibility to simulate hybrid systems in the Matlab framework is 
provided by the SiMULiNK software package. SiMULiNK is a graphical, block- 
diagram oriented tool for modeling, simulating, and analyzing linear and nonlin- 
ear dynamic systems which is based on Matlab ’s representations and numerical 
routines. The latest release of the package introduces a toolbox called State- 
Flow which makes it possible to introduce discrete event systems as blocks in a 
SiMULiNK block diagram. This new feature makes SiMULiNK particularly inter- 
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esting for hybrid system simulation and was the motivation to include the tool 
in this comparison. 

StateFlow uses statccharts [Har87] as the modeling paradigm to describe 
the behavior of a block. Technically, each StateFlow block is a masked SlMU- 
LINK model. This means, StateFlow transforms each statechart into the SlMU- 
LlNK-specific internal representation, a so-called S-function. This S-function is 
the agent SiMULiNK interacts with for simulation and analysis. The control be- 
havior represented by the statechart complements the algorithmic behavior mod- 
eled by the standard block by switching variable values. Obviously, this makes 
it possible to switch continuous behaviors during a simulation run depending on 
state events. 

Modeling the two tanks example is straightforward in SiMULiNK. We first 
implemented the plant dynamics by introducing blocks for the mass balances 
of the two tanks and the characteristic curves of the valves. The autonomous 
switching of the model at ^2 = 0.39 m was realized by a ’switch’ block in the 
feedback loop from h\ and /12 to the input of the two tank balances. The easi- 
est part was to build the model of the control program. Since the given SFC is 
merely a sequential machine and does not include any parallel behavior, it can 
be transformed into an isomorphic statechart (see Fig. 6). Coupling this State- 
Flow block to the other blocks via the valve commands and the level /12 gives 
us the desired hybrid model of the system. 

Correct state event detection during the integration of switched continuous 
models can be enforced in SiMULiNK by use of the so-called ‘hit crossing’ block. 
It forces the simulation to locate zero crossings of the input signal. The block 
detects the zeroes (with machine precision) by choosing a smaller step-size (in the 
neighborhood of the crossing point) for the integration progress. We employed 
two of these blocks for detecting the crossing of the thresholds L_plus and 
L_minus. 

Care has to be taken when real-time constraints arc spccihcd in the state- 
chart of a StateFlow block, e.g., activity durations as in our example. For the 
simulation, the time steps in StateFlow are different from the time steps used 
for the standard continuous SiMULiNK blocks. To realize a proper synchroniza- 
tion, the variable systemtime has to be fed into the StateFlow block as an 
additional input. 

In conclusion, the addition of StateFlow to SiMULiNK provides a com- 
fortable environment for modeling and simulating of hybrid dynamical systems. 
When applied to the two tanks example, the modeling step was straightforward 
whereas the simulation required some experience with respect to the synchro- 
nization of the statechart and the standard blocks. 

3.3 gPROMS 

The development of the gPROMS package (general PROcess Modeling System) 
[PB94] was initiated in 1988 at Imperial College, London, and is currently still 
ongoing. The tool can be regarded as a successor of the now commercialized 
package SpeedUp which is the most used simulator in the processing industries. 




172 



S. Kowalewski et al. 



Steuerungl 







Fig. 6. Control program TwoTanks as a statechart in StateFlow 



gPROMS also aims at process engineering applications and provides a general- 
purpose modeling environment for the entire range of processes from purely 
continuous to purely batch. In addition, capabilities for dynamic and steady-state 
simulation, optimization and visualization are included. Particular attention has 
been paid to handle modeling complexity by the concept of hierarchical submodel 
decomposition. 

gPROMS distinguishes between two types of modeling entities: models and 
tasks. Models describe the physicochemical mechanisms governing a plant be- 
havior formulated as integral, partial and ordinary differential, or algebraic e- 
quations, whereas tasks characterize the operating policies and control strategies 
imposed on the process. The tasks and models are combined to a process which 
delivers thus a description of the controlled system. 

Modeling the two tanks plant, the mixed set of algebraic and differential 
equations as well as its switching discontinuities are defined in a model. Since 
the syntax of the gPROMS modeling language permits the use of if-then-else- 
as well as case-structures, the code for the description of the plant’s behavior 
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can be comfortably generated. The operating procedure of the plant consisting 
of four different steps is expressed in four tasks. Each task is triggered by the 
switching of a global mode variable which observes the two tank levels. Both 
modeling entities form the whole process in which the sequential execution of 
the tasks is prescribed. 

For the simulation, a relative accuracy of le-6 has been set to restrict the 
local truncation error, and a monitoring step length of 1 second has been chosen. 
The simulation results have been visualized with the gRMS ( gPROMS Results 
Management System) facility. A typical simulation result can be seen in Fig. 7. 
It shows the overflow of the second tank for parameter set 2. Again, no limits 
for the heights were necessary in the model. 

However, if only simulation is used to treat problems with nondeterministic 
aspects, the right choice of the considered cases is left to the user’s intuition. 
Dimitriadis et al. suggested an approach in which the worst case scenarios are 
detected automatically [SPD95]. For this purpose, the process model is aug- 
mented by binary variables assessing the systems safety in every time step of the 
system’s possible trajectories. With the gPROMS optimization facility gOPT, 
a worst case of the system’s behavior can then be found by a minimization or 
maximization (depending on the safety evaluation strategy) of the sum of all 
binary variables. 

In general, the modeling of hybrid systems with gPROMS is comfortable. 
However, there are still only little diagnostic facilities to trace simulation fail- 
ures back and solve them. The possibility of debngging and on-line variable 
monitoring is limited. As a conclusion, it can be said that gPROMS is well 
suited for the two tanks example, although gPROMS offers no facilities for a 
sound verification of the control program. The proof of fulfilment of the specified 
plant behavior requirements for the chosen sets of parameters could be given by 
identifying and simulating the corresponding worst-case situations. 



3.4 Shift 

Shift (Hybrid System Tool Interchange Format) is a specification langnage and 
simulation tool for hybrid systems, currently developed at Berkeley [DGS96]. 
The language allows an object-oriented modeling of multicomponent systems 
with hybrid behavior, i. e. 'components^ are instantiated from predefined class- 
es, called 'types', and features as the inheritance of properties are available. 
The overall model {'world') is build from the set of components which inter- 
act via input and output variables, and event synchronization. Though some 
modifications referring to the connnection and synchronization of components 
have been introduced [DGV97], the theoretical basis for the description of the 
components’ behavior is the Hybrid Automata paradigm defined by Alur et al. 
[ACH+95]. Hence, a component’s continuous evolution is given by a DAE-System 
(the 'flow') and the discrete dynamics by a finite state automaton, where event 
labels, guards, invariants and actions can be assigned to the transitions. The 
world behavior follows from the composition of all component’s evolution. 
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Heights (m) Valve position (-) 




Time (sec) 



?W0_TMKS. TANKS. HI 
7W0_TANKS. TANKS. H2 
?W0_TANKS. TANKS. PI 
TWO TANKS. TANKS. P2 



Fig. 7. Visualization of the simulation results by gRMS (parameter set 2) 



In difference to the other tools used in this comparison, Shift enables the 
user to model and simulate dynamically reconfigurable hybrid systems: An action 
assigned to a discrete event of one component can trigger the creation of a 
new component or the rearrangement of the connection between components. 
This allows to describe networks which change their structure at run-time, as it 
appears for example when modeling automated traffic systems. 

The Shift tool comprises a compiler to translate Shift specihcations to C 
and to generate an executable simulation hie. Furthermore, it offers an online- 
debugger and a data visualization and animation facility (TkShift) to carry 
out simulation. Starting with an initialization phase, a simulation runs by the 
stepwise execution of continuous and discrete state evolutions in the order of 
their occurrence. Using the Shift version 1.8 for this investigation, only a 4‘^- 
order Runge-Kutta-method with hxed step-size was available to evaluate the 
continuous evolution. If the simulation generates several discrete events at one 
time instant, these are executed in a hrst-in-hrst-out strategy, which means, 
no user-specihed interaction is possible in the case of non-deterministic discrete 
behavior. 
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To model the two tanks example, we created components for each of the 
system's technical devices (tanks, valves, connecting pipes) as well as for the 
supervising controller. The links between these eomponents were constituted 
by defining the continuous quantities (liquid levels, flowrates, valve positions, 
etc.) as input /output variables and setting the connections correspondingly in 
a superordinated block. The components’ discrete states are naturally given 
by discretizing the range of the continuous variables according to thresholds 
being important for the control objective (e. g., ’opening’, ’steady’, ’closing’ for 
a control valve). A transition between two of these discrete states is introduced 
for the case that the continuous variable crosses the separating threshold in 
one direction. For the ’controller’ component, a reset of the variables denoting 
the valves’ mode (’opening’, ’closing’, ’steady’) is assigned to the transitions as 
action. For components inhering continuous dynamics, appropriate algebraic and 
differential equations define the flow within the discrete states. Specifying the 
two tanks system in this manner, TkShift produces plots as shown in Fig. 8 
for the case of a periodic switching of the outlet valve (Pl_n, P2_n denotes the 
normalized setting of the valves Vi and V 2 : it equals 1 if closed and 0 if opened) . 



" h1(TanksO) 
•vs>TimeClick 
“ h2(TanksO) 
-vs-TimeClick 

— P1_n(Valve10) 
•vs-TimeClick 

— P2_n(Valve20) 
-vs-TimeClick 




200 400 600 



Time 



Fig. 8. The simulation results visualized by TkShift (parameter set 4) 



When applying Shift to the example with the different parameters and 
modes, we found that Shift is suitable to model processes of this type whereas 
some of its main features, as the dynamic reconfiguration or the inheritance of 
the components’ properties, were not used. Some restrictions are given by the 
SHiFT-syntax, e. g. no if-then-else-structures are permitted in the flow definit- 
ions. Furthermore, the simulation algorithm does not offer the versatility of tools 
like gPROMS, Dymola or SiMULiNK - e. g., no facility for an accurate detec- 
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tion of state events while calculating the continuous state evolution is available. 
Since Shift does not offer any verification procedures, tasks like analyzing the 
example control program in the disturbed case can only be solved by worst-case 
simulation. 

It shall be mentioned that in newer versions of Shift some modifications to 
lessen or remove the disadvantages stated have been introduced: A variable step- 
size integration algorithm is available now, such that the integration precision 
should no longer considerably differ from that of the Matlab tool or gPROMS. 
Additionally, in [GKV97] an interpolation method for a more accurate detection 
of state events is announced. 

3.5 Dymola 

Dymola is a general-purpose modeling and simulation environment. It was de- 
veloped in 1978 at Lund Institute of Technology in Sweden [Elm93] and has been 
distributed commercially by Dynasim AB since 1993. The most relevant concepts 
realized in Dymola are object oriented modeling and non-causal modeling. 

Object oriented modeling allows the independent definition of partial mod- 
els, the construction of model libraries and the hierarchical model composition. 
These tasks are supported in Dymola by a graphical model editor and browser. 
A new model can be built by defining its equations and/or connecting already 
existing modules graphically. 

Basic modules need not to be specified by any program language, instead 
the only thing to do is to declare the variables and parameters, to write down 
the equations, to define the interface vectors and to create an object symbol. 
Because of the object oriented concept, Dymola is suitable for multi domain 
modeling. Subsystems of different domains, for example electrical, mechanical 
and control systems, which are modeled with domain specific symbols, can be 
connected to an unique model. 

Non-causal modeling means that it is possible to give the equations which 
describe the system behavior in any form. It is neither necessary to specify input 
or output variables nor to convert the equations to assignment statements. Even 
implicit equations can be specihed. Therefore the interfaces of the submodel 
blocks arc in general neither inputs nor outputs and feedbacks need not to be 
considered by additional connections. 

When the model is being transformed into a set of equations, algebraic equa- 
tions are inserted for each connection. This corresponds to Kirchhoff’s current 
law. Symbolic equation processing is used to obtain the best possible assignment 
sequence, which makes the simulation more efficient. The DAE system is con- 
verted to state-space form if possible or to reduced DAE form. The non-causal 
modeling in combination with the symbolic equation processing enables both 
more simple models and more efficient simulation. 

Discrete components can be modeled on basic language level by if-then-else 
expressions or by when statements. A limiter for example can be written as: 
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y = if u>Highlimit then Highlimit 

else if u<Lowlimit then Lowlimit 
else u 

A when statement has the following general structure: 

when <condition> then 
<assignments> 
end when 

Dymola translates these basic mathematical elements into time and state 
events when generating code. Modeling the two tanks example was realized by 
defining two subsystems: the controller and the physical system. The controller 
was modeled as a Petrinet (Fig. 9) using the components of the included Finite 
State Machines and Petrinet library which is built on basic discrete language 
elements. 



@ >l> 0 ►[> (ff (ff >!► (j) 



Time>Time1 Time>Time2 



Level>L_plus Level<L_minus 



Fig. 9. Control program two tanks as a Petrinet in Dymola. 



Additional Boolean expressions are necessary to derive the control signals 
to the valves from the states of the Petrinet. The physical system contains the 
equations from Section 2 without any alterations using if-then-else expressions. 
Only Eq. 3 needed to be modified to 

der(Pl) = if PKO and V_1 then 0 

else if Pl>80 and not V_1 then 0 
else if V_1 then -1 
else +1 

To summarize, Dymola is well suited to treat the two tanks example. How- 
ever, it cannot take advantage of its symbolic equation processing because the 
example is not complex enough. 

3.6 BaSiP 

As opposed to the other tools in this comparison, BaSiP (Batch Emulation 
Package) is not a general-purpose modelling and simulation tool, but designed for 
a specific application domain: recipe driven chemical batch processes [WFSE96]. 
Though this example is not exactly what is generally understood as a batch 
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process, with some modelling tricks and some extensions to the program it is 
possible to model and simulate this system. 

The key idea of BaSiP is to provide a flexible and open software architec- 
ture that allows to integrate different components for modelling, simulation, and 
visualization [FE97]. 

By now, BaSiP consists of a graphical user interface to model plant and 
recipes separately, a choice of different simulators, and several interfaces to visu- 
alization programs (e. g. GNUplot, see Fig. 10). An animation of the simulation 
run is also possible using the graphical editors. In a configuration shell, sim- 
ulation experiments can be configured by simply putting together the desired 
models and components. 

On the simulator side, two different approaches to simulate hybrid systems 
(of which batch processes are a typical example) are integrated. The first uses a 
system of ODEs that can be switched. Special attention has been given to event 
detection algorithms and a dynamic reconfiguration of the state vector after each 
switching to minimize the number of currently active equations. Another, novel 
approach is based on the discrete-event simulation paradigm. Here, the process 
description is simplified using algebraic equations that are evaluated to predict 
the next event in the simulation. 

If the process is viewed from a sufficiently high level, this approach can 
produce results with no loss of accuracy compared to continuous simulation 
but in orders of magnitude less computation time. For this example, however. 




Fig. 10. The simulation results of BaSiP visualized by GNUplot (parameter 
set 3) 
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the process is viewed on a level of detail that is actually above the scope of 
BaSiP. Thus, with the simplifications taken in the discrete-event approach, no 
satisfactory results can be achieved. Only the continuous approach, extended by 
some new model blocks, can handle this system. 

The sequence control of the example can easily be translated into a BaSiP 
recipe, as both use the same representation as a sequential function chart. How- 
ever, the semantics of a recipe are slightly different to that of sequence control 
because recipes lack the notion of state that can be set and reset. To represent 
the “automatic” flow operations between the tanks (flow takes place although 
no corresponding recipe phase is active), parallel branches with dummy filling 
operations were included in the recipe. 

Because modelling in BaSiP normally takes place on a higher level than re- 
quired here, some model blocks had to be extended, while some new ones had 
to be added. This includes valves that have limited positioning speed or level- 
dependent flow operations. The disturbed case requires additional modifications 
of the models, and could not been implemented so far. Modifications or addi- 
tions of model blocks require changes in the source code, written in C++, thus 
making such changes quite inconvenient. Future releases shall enable at least 
minor modifications without these efforts. 

As a summary, it can be said, that BaSiP proves to be apt even for cases it 
is not designed for, but lacks of flexibility to adapt the program to special cases. 

3.7 Comparison of numerical results 

In this section, we take a closer look at the detailed numerical results for the two 
tanks example provided by the different simulators. Since this case study inves- 
tigates the suitability to hybrid systems, our special interest is in the handling of 
discrete phenomena. In particular, we look at how accurate state events, i.e. the 
switching point and the corresponding time, are determined. For this purpose, 
the fourth parameter set is chosen because in this case the state events “level /12 
reaches the threshold L_plus or L_minus, resp.,” occur periodically (cf. Fig. 7). 
This allows us to examine how the error can increase by accumulation during 
multiple occurence of state events. 

Table 2 shows the results of the investigation. It presents the computed times 
when ft -2 reaches L_minus for the first (1st event) and the tenth time (10th event), 
together with the corresponding value of /12 • The main conclusions which can be 
drawn from it are the following. 

— It is reasonable to derive from the rows 3, 9 and 10, which represent the most 
accurate simulation runs of three of the tools with state event detection, that 
the correct values have to be near to 303.13 s for the first and 2669.9 s for 
the tenth event. 

— As expected, the best results (relative error < le-4) are achieved when inte- 
gration routines with step-size adaption, state event detection mechanisms 
and sufficiently small tolerances are used. However, the rows 1 and 2 show 
that also a fixed step-size can lead to satisfying results when event detection 
is employed. 
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— If neither variable step-sizes nor state event detection is available, the errors 
are of greater magnitude (cf. rows 4, 5, 12 and 13). In these cases, it is 
necessary to decrease the step-size which will increase the computational 
effort. 

— The results of Simulink/StateFlow are worse than expected, in partic- 
ular row 7. This may be due to the problem of initializing the interaction 
between the statechart for the eontroller and the Simulink model for the rest 
of the plant. We found no way to define the values of the exchange variables 
(opening/closing the valves) for the first integration step. As a workaround, 
the initial step was set to le-7 (which was only possible for the variable 
step size iteration). However, this does not explain the relatively large errors 
(magnitude le-3) in rows 6 and 7 and that the time value of the 10th event 
is worsened by use of the hit crossing blocks. 









1st event 


2nd event 


# 


step-size 


tolerance 


time [s] lvalue [m] 


time [s] lvalue [m] 



Mat lab /Taylor (4th order Rungc-Kutta) 



1 


1 


- 


303.141 


.300000 


2670.5 


.300000 




0.1 


- 


303.133 


.300000 


2670.0 


.300000 


T 


var, max 5 s 


le-6 


303.127 


.300000 


2669.9 


.300000 


Simulink / StateFlow (Dorman-Prince) 
(a) Without hit crossing blocks 


4 


1 


- 


307 


.299260 


2711 


.299378 


5 


0.1 


- 


303.5 


.299936 


2671.4 


.299998 


6 


var, max 1 s 


rel. le-6, abs. le-15 


303.76 


.300000 


2670.50 


.300000 


(b) With hit crossing blocks 




var, max 1 s rel. le-6, abs. le-15 303.66 .300000 


2666.14 


.300000 



gPROMS (Backward Differentiation Formula) 



8 


var 


rel. le-4, abs. le-4 


303.21 


.300000 


2670.82 


.300000 


9 


var 


rel. le-6, abs. le-15 


303.13 


.300000 


2669.87 


.300000 



Dymola (Adams/Bashforth/Moulton combined with Gear) 



10 


var 


le-6 (rel. and abs.) 


303.126 


.300000 


2669.88 


.300000 



BaSiP (Euler) 



0 


|var, max 1 s 


1 le-8 


|303.849| 


.300000 


|2673.66| 


.300000 1 


Shift (4th order Runge-Kutta) 


12 


1 


- 


301 


.299464 


2695 


.299510 


13 


0.1 


- 


302.3 


.299947 


2666.1 


.299944 



Table 2. Numerical results of specific simulation runs 



Of course, the values of Table 2 have to interpreted with caution. It was 
impossible to provide absolute equivalent conditions for a comparison. The tools 
differ too much in many respects, e.g. available integration routines (the rela- 
tively weak results of BaSiP in row 11 are due to the fact that only an Euler 
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routine is implemented) and possibilities to adjust maximal and minimal step- 
sizes and tolerances. Additionally, the execution of the simulation programs is 
different. In some tools an interpreter processes the code, other tools generate 
C code which has to be compiled and executed externally. For this reason, CPU 
times are not included in Table 2. They were in the range of seconds for each of 
the tools except for Taylor’s simulator which uses the Matlab interpreter and 
therefore was much slower. 

4 Application of the verification tools 

4.1 HyTech 

HyTech is a model checker for hybrid systems [HHW97]. To analyze systems 
with HyTech they must be modeled as Linear Hybrid Automata [ACH+95]. 
For this example a subclass of hybrid automata, namely Timed Automata ( TA) 
was chosen, although a system description with Linear Hybrid Automata would 
have been less complex and more accurate. Due to a known integer overflow 
problem in the analysis of hybrid automata, TA turned out to be much more 
suited in terms of feasible complexity. The model of the two tanks system is 
illustrated in Fig. 11. It consists of three TA, one describing the levels in the 
two tanks (Levels), one describing the position of the outlet valve (Valve2) and 
one for the behavior of the controller (Controller). The valve between the two 
tanks was introduced with a constant open position and a leak in the system was 
not considered. The analysis of the model can now be used to reveal whether or 
not the controller manages to avoid an overflow in T2 for given starting levels 
LI and L2. In Fig. 11 the model of the two tanks system is illustrated. The 
controller cycles between four locations, opening and closing V2 whenever the 
level in T2 hits the minimum/maximum landmark Ljminus/ Ljplus. The TA 
Valve2 changes its locations (which represent the discrete position of the valve) 
according to a clock which captures the dynamics of the valve. It propagates a 
change of the discrete state on to the TA Levels and reacts to open/close signals 
from the TA Controller. Each location in Levels represents a triple of discrete 
values for the levels in T1 and T2 and the position of U2. A clock captures the 
dynamics of the possible change of both levels according to the current levels and 
position of U2. To create a TA for the continuous behavior the given differential 
equations had to be discretized. This was done based on simple approximation 
method which maps the continuous state space into a set of discrete points that 
yield the locations of the discrete model. Then the successors of each location are 
evaluated by extending the vector of the state variable derivatives up to the next 
plane with discrete points. Then, all points directly adjacent to this intersection 
point are taken as the successors. 

The size of such a discrete model depends on the number of discrete levels 
considered for the two tanks {Nti, Nt 2 ) and the number of discrete positions 
considered for the outlet valve {Nv 2 )’ To derive detailed models of the system a 
Matlab function was implemented, which takes three parameters, namely the 
numbers of discrete points given by Nti, Nt 2 , and Nv 2 , and then generates 
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o-^o-^o-^o-*-o-*>o 

n a n n n r; 

o^o<-o^o<-o^o 



Fig. 11. Timed automata model of the Two Tank System 



a HyTech input file with the corresponding TAs. Another Matlab function 
reads the HyTech output and generates a graphical representation of the set 
of reachable locations. The most detailed model which HyTech could still an- 
alyze on a Hypersparc with 100Mb RAM was derived with = Nx 2 = 40 
and Nv 2 — 20. The resulting TA for the representation of the levels in the two 
tanks had more than 30.000 locations. The set of 2.100 reachable locations for 
this model from the initial levels LI = 0.7m and L2 = 0.8m with landmarks at 
L-plus = 0.8m and Ljminus = 0.2m, is given in Fig. 12. The figure reveals that 
the level in Tank 2 does not exceed the 40th discretization point which corre- 
sponds to Im. The results show that HyTech is in general suited for problems 
like the two tanks example, although it soon exhibits its complexity limits. Still, 
more accurate results could be achieved with less modeling effort, if HyTech ’s 
analysis algorithms performed better on linear hybrid systems. 

4.2 SMV 

SMV (Symbolic Model Verifyer) is a tool for applying the verification tech- 
nique Symbolic Model Checking [CGL94]. The SMV-System provides a lan- 
guage which allows to describe the considered transition system hierarchically. 
The model checker uses an Ordered Binary Decision Diagrams (OBDD) based 
search algorithm to check the specification of the desired behavior given as a 
Computation Tree Logic (CTL) formula. If the system does not show the de- 
sired behavior, SMVproduces an execution path with a counter-example. 
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Fig. 12. Graphical representation of the reachability set determined by HyTech 



SMV has been applied to the verification of logic process controllers by Pow- 
ers et al. [TPP97], The two tanks example differs from those applications in that 
the continuous dynamics are much more complex here. Just as in the HyTech 
case, a discrete approximation has to be derived form the continuous model to 
be able to verify the two tanks system with SMV. Again, this was implemented 
by means of Matlab. Here, the approximation algorithm partitions the contin- 
uous state space into boxes and calculates the possible residence times in each 
box from the differental equations. All subsystems, like the valves, the controller 
and the discretized liquid levels are modeled separately. 

In contrast to HyTech, SMV does not offer the possibility to model quan- 
titative time information. Therefore the time had to be introduced as additional 
automata representing clocks which take one transition at each SMV execution 
cycle. Since SMV does not provide a synchronisation concept in the sense that 
transitions can be forced to be executed simultaneously, we keep an image of 
the discrete state space with the actual state always one time step behind. By 
this trick, events can be detected by evaluating conditions on the states in two 
systems. 

The size of the SMV input file generated by Matlab, of course, grows with 
the discretization grade. Since SMV imposes a restriction on the input file size, 
it could be applied to the two tanks problem only for very rough discretizations. 
As a consequence, fine points of the analysis, like the behavior for the fifth 
parameter set, could not be determined correctly. 
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5 Discussion 

We posed a benchmark problem to verify requirements for a simple but real 
plant controlled by a logic control program and applied eight tools for simu- 
lation or verification of hybrid systems to it. To summarize shortly, it can be 
stated that due to the relatively complex continuous dynamics the two ver- 
ification tools, HyTech and SMV, had major problems. Although automatic 
abstraction techniques were developed successfully, both tools soon reached their 
complexity limits (SMV much earlier than HyTech due to the additional dis- 
cretization of time). On the other hand, all six simulation tools (Taylor/MATLAB, 
Simulink/StateFlow, gPROMS, Shift, Dymola, and BaSiP) could solve 
the problem satisfactorily, but the user has to rely on his/her intuition to identify 
the worst cases when it comes to nondeterministic occurences of disturbances. 
To discriminate further, Shift lacks of an exact identification of state events. 
Taylor's Matlab toolbox and BaSiP could only be applied after some pro- 
gramming for which deeper knowledge of the tool was necessary. SiMULiNK with 
StateFlow probably is the most comfortable environment for the user in this 
comparison but it revealed accuracy problems. For gPROMS and Dymola the 
two tanks example is an easy task to solve. Both tools could not show their 
particular strengths. 

Of course, our choice of tools for this case study is incomplete. Many sim- 
ulation packages are missing. With respect to verification, not only other tools 
can be applied, it is also possible to use different methods for discretization of 
the continuous dynamics. We hope that our contribution will motivate other 
researchers to apply their tools and methods of interest to this example and, by 
doing that, make the comparison more comprehensive. 
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Abstract. The theory of formal verification is one of the main approach- 
es to hybrid system analysis. Decidability questions for verification algo- 
rithms are obtained by constructing finite, reachability preserving quo- 
tient systems called bisimulations. In this paper, we use recent results 
from stratiheation theory, subanalytic sets, and model theory in order to 
extend the state-of-the-art results on the existence of bisimulations for 
certain classes of planar hybrid systems. 



1 Introduction 

Hybrid systems consist of finite state machines interacting with differential equa- 
tions. Various modeling formalisms, analysis, design and control methodologies, 
as well as applications, can be found in [3,4,5,10,17]. Formal verification is the 
main computational approach for analyzing properties of hybrid systems. One of 
the most important verification problems for hybrid systems is the reachability 
problem which asks whether trajectories of the hybrid system can reach certain 
undesirable regions of the state space. Since hybrid systems have infinite state 
spaces, the decidability of verification algorithms is very important. 

A uniform framework for tackling the decidability issue is provided by the 
notion of hisim,ulation. Bisimulations are quotients of the original hybrid system 
that are reachability preserving. Showing that an infinite state hybrid system has 
a finite state bisimulation is the first step in proving decidability. This approach 
has been successfully applied to timed automata [2], multirate automata [1], and 
initialized rectangular automata [19,12]. It should be mentioned that the notion 
of bisimulation is closely related to the various consistency notions for discrete 
and continuous systems [7,8,18]. 

Since the discrete dynamics are already finite, it is clear that decidability 
results for hybrid systems depend crucially on the success of obtaining hnite 
bisimulations for continuous dynamics. The cases considered so far in the lit- 
erature dealt with simple dynamics: x = I for timed automata [2], i: = a for 
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and DAAH 04-96-1-0341. 



P. Antsaklis et al. (Eds.); Hybrid Systems V, LNCS 1567, pp. 186—203, 1999. 
© Springer- Verlag Berlin Heidelberg 1999 




Hybrid Systems with Finite Bisimulations 



187 



multirate automata [1], i 6 [a, b] for rectangular automata [19], and Ax for 
linear hybrid automata [11]. In this paper, we extend the bisimulation method- 
ology to hybrid systems with more general dynamics. We first present the s- 
tandard bisiniulation algorithm which, upon termination, provides the desired 
finite bismilarity quotient. In [15], we used purely geometric methods to show 
that the bisimulation algorithm terminates for a class of hybrid systems with pla- 
nar linear dynamics. In this paper, we combine mathematical techniques from 
differential geometry and recent results in logic model theory in ordet to prove 
existence of finite bisimulations for various new classes of hybrid systems with 
planar continuous dynamics. This convergence of mathematical logic and differ- 
ential geometry also provides a natural framework for extending the decidability 
frontier for more general classes of hybrid systems. Such extensions will require 
pushing the boundary of decidable theories in mathematical logic. 

Abstracting a discrete graph from a hybrid system requires the analysis of 
trajectories of vector fields and their intersection properties relative to a given 
collection of sets. Considering hybrid systems with arbitrary dynamics and ar- 
bitrary state partitions would soon lead to pathological situations. Subanalytic 
sets [6,13,21] provide a rich class of sets which have many desirable local inter- 
section properties with trajectories of analytic vector fields. Subanalytic sets can 
also be partitioned into smooth embedded submanifolds in a form suitable for 
constructing a bisimulation. Such partitions are called stratifications. Moreover, 
we show that relaxing the class of vector fields or sets in some naive ways leads 
to pathological situations. On the other hand, the concept of o-minimal theo- 
ries in logic [24,25,26] identifies classes of sets with good intersection properties 
suitable for the global study of trajectories of vector fields. The combination of 
techniques from both fields highlights the kind of properties of sets that play a 
central role in obtaining discrete abstractions. 

The outline of the paper is as follows: In Section 2 wc review the notion of 
bisimulations of transitions systems. In Section 3 we dehne the class of hybrid 
systems under study and describe the main bisimulation algorithm. Section 4 
presents some basic facts about stratiheation theory and subanalytic sets and 
relates them to the construction of bisimulations. In Section 5 we present recent 
results in model theory which are used in Section 6 in order to obtain classes 
of systems for which the bisimulation algorithm terminates. Section 7 contains 
conclusions and issues for further research. 

2 Bisimulations of Transition Systems 

A transition system T = (Q, A, Qo, Qf) consists of a (not necessarily hnite) 
set Q of states, an alphabet S of events, a transition relation -^Q, 

a set Qo —>Qoi initial states, and a set Qf of hnal states. A transition 
(gi,<T, g 2 ) is denoted as gi qa- The transition system is hnite if the 
cardinality of Q is hnite and it is inhnite otherwise. A region is a subset P ^Q. 
Given cr G A we dehne the predecessor Prea-(P) of a region P as 

Pre„{P) = {g G Q I G P and g p} 



( 1 ) 
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Given an equivalence relation —>Q. we define a quotient transition system 

tj ^as follows: Let Q/^denote the quotient space. For a region P we denote by 
Pj the collection of all equivalence classes which intersect P. The transition 
relation on the quotient space is defined as follows: for Qi,Q2 6 Q/ — 
Qi Q2 iff there exist qi G Qi and (72 G Q2 such that qi q2- The quotient 
transition system is then T / [Q/ if, Qo/ ^,Q f/ 

Given a partition ^on Q, we call a set a ^block if it is a union of equivalence 
classes. The partition ^is a bisimulation of T iff Qo,Qf are ^blocks and for 
all a £ S and all ^blocks P, the region Precr{P) is a ^block. In this case the 
systems T and T/ ^ are called bisimilar. A bisimulation is called finite if it has 
a finite number of equivalence classes. Bisimulations are very important because 
checking reachability on the bisimilar transition system is equivalent to checking 
reachability of the original transition system [ 11 ]. Therefore, if T is infinite and 
T/ ^is a finite bisimulation, then verification algorithms for infinite systems are 
guaranteed to terminate. If ^ is a bisimulation, it can be easily shown that if 
p ^q then 

B1 p e Qp iff (7 G Qf, and p G Qo iff g G Qo 

B 2 if p p' then there exists q' such that q ^ q' and p' -^q' 

Based on the above characterization, given a transition system T, the following 
algorithm computes increasingly finer partitions of the state space Q. 

Algorithm 1 (Bisimulation Algorithm for Transition Systems) 

Set Q/ ~^= {Qo n Qp, Qo \ Qfi Qf \ Qo,Q \ {Qo U Qf)} 
while 3 P,P' G <5/ ^and a e S such that 0 ^ P D Pre„{P') / P 
set Pi = P n Preo-(P'), P2 — P \ Pre^{P') 
refine Q/^ (Q/^\{P|) U {Pi, P2} 
end while 

If the algorithm terminates, then the resulting quotient transition system is a 
finite bisimulation. The goal of the next sections is to show that the above 
algorithm terminates for transition systems generated by a class of planar hybrid 
systems. 

3 Bisimulations of Hybrid Systems 

In this paper, we focus on transition systems generated by the following class of 
hybrid systems. 

Definition 1. A hybrid system H = {X, Xq, Xp, F, E, I ,G, R) consists of 

— X — Xu —fXc is the state space with Xp — {gi, . . ■ , gn} (md Xc an analytic 
manifold. 

— Xq ~^X is the set of initial states 

— Xp ^X is the set of final states 

— F : X TXc assigns to each discrete state q ^ Xp an analytic vector 
field F{q, > 
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— E is the set of discrete transitions 

— I ■. Xd assigns to each discrete state a set I{q) Xq called the 

invariant. 

— G : E (8^ Xo—Q^^^ assigns to e = {qi, q2) ^ E a guard of the form {qi}^U , 
U^I{qi). 

— R : E Xi)^ 2 ^^ assigns to e ^ ((71,(72) G E a reset of the form {q2}^V, 
V^I{q 2 ). 

Trajectories of H start at any [q, x) S Xq and consist of continuous evolutions or 
discrete jumps. Continuous trajectories evolve according to the continuous flow 
F{q, ^as long as the state remains in the invariant set I{q). If the flow exits I{q), 
then a discrete transition is forced. If, during the continuous evolution, the state 
(q,x) enters a region G(e) for some e G E, then discrete transition e is enabled, 
and the state may instantaneously jump from {q,x) to any [q' ,x') G R{e). Then 
the system evolves according to F{q', ^ Notice that the discrete transitions 
allowed in our model are similar to the type allowed in initialized rectangular 
automata [ 19 ]. Finally, we assume that our hybrid system model is non-blocking, 
that is from every state either a continuous evolution or a discrete transition is 
possible. 

Every hybrid system H = [X, Xq, Xp , F, E, I ,G , R) generates a transition 
system T = (Q,X,^,Qo,Qf) by setting Q ^ X, Qo = Xq, Qp = Xp, X = 
E U {t}, and (Uees “^)U ^ where 

Discrete Transitions {q,xf) [q',x') for e G E iff {q,x) G G(e) and {q',x') G 
R{e) 

Continuous Transitions (qi,a:i) ^ (92, 2:2) iff <?i = 92 and there exists 5 ^0 
and a curve x : [0, <5] M with x(0) = xi, x(d) = X 2 and for all t G [0, 5] 
it satisfies x' = F{qi,x{t)) and x{t) G I{qi). 

The continuous r transitions are time-abstract transitions, in the sense that 
the time it takes to reach one state from another is ignored. Having defined 
the continuous and discrete transitions ^ and ^ allows us to formally define 
Prer{P) and Pree{P) for e G E and any region P -^X using ( 1 ). Furthermore, 
the discrete transitions allowed in our hybrid system model result in 



Pree(P) 



0 if P n R{e) = 0 
G(e) if F n R{e) / 0 



( 2 ) 



for all discrete transitions e G E and regions P. Therefore, if the sets R{e) and 
G(e) are blocks of any partition of the state space, then no partition refinement 
is necessary in the bisimulation algorithm due to any discrete transitions e G 
E. This fact, decouples the continuous and discrete components of the hybrid 
system as long as the initial partition in the bisimulation algorithm contain the 
invariants, guards and reset sets, in addition to the initial and final sets. This 
allows us to carry out the bisimulation algorithm independently for each discrete 
state. 
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More precisely, define for any region P — and q G X[j the set Pq — {x G 
Xc '■ [q,x) G P}. For each discrete state q G Xjj consider the finite collection of 
sets 



A = {/(g), (Xo)„ (X^) J U {G{e)q, R{e)q : e G E} (3) 

which describes all the relevant sets associated with discrete state q. Let Sg be 
the coarsest partition of Xc compatible with the collection Aq (by compatible 
we mean that each set in Aq is a union of sets in Sq). The (finite) partition Sq 
can be easily computed by successively finding the intersections between each of 
the sets in Aq and their complements. These collections Sq will be the starting 
partitions of the following bisimulation algorithm for hybrid systems. 

Algorithm 2 (Bisimulation Algorithm for Hybrid Systems) 

Set X/^^UqSq 
for q G Xd 

while 3 P,P' G Sq such that 0 P D PreT-(P') P 
Set Pi = P n Prer{P')\ P 2 = P \ Pre^{P') 
refine Sq = {Sg \ {P}) U (Pi, P 2 } 

end while 
end for 

A few comments are in order here. The key problem is to investigate how 
the flow of F{q, ^interacts with the sets Sq for a single discrete state q. This 
requires that the trajectories of the vector field F{q, ^have “nice” intersection 
properties with such sets. Since the goal is to obtain finite partitions, it will 
become important that we restrict the study to classes of sets with good “finite- 
ness” properties, for example, sets with finitely many connected components. In 
the subsequent sections we identify classes of sets and vector fields which exhibit 
such properties and for which Algorithm 2 terminates. 

One can also view the partitions in the algorithm as a way of discretizing 
the system trajectories. This suggests studying the continuous transitions by 
looking only at the points at which the trajectories move from one set in Sq to 
an “adjacent” one. This is in general not possible because sets could have rather 
pathological boundaries (see also Example 4). We will see in the next section 
that subanalytic sets are free from such pathologies and that in fact one can 
formalize the idea of trajectory discretization associated to the partition in that 
case. 

We conclude this section with an example that shows that, even in apparently 
simple situations. Algorithm 2 does not terminate. 

Example 1. Let F be the linear vector field x on Assume the 

partition of consists of the following three sets (see Figure 1): Pi = {(x, 0) : 
0 ^4}, P 2 = {(x, 0) : ®4 ^x < 0}, P 3 = \ (Pi U P 2 ). The integral 

curves of F are spirals moving away from the origin. The first iteration of the 
algorithm partitions P 2 into P 4 = P 2 n Prer{P\) = {(x.', 0) : xi -^x < 0} and 
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Fig. 1. Algorithm 2 does not terminate 



P 2 \ Prer{Pi)- Here Xi < 0 is the x-coordinate of the first intersection point 
of the spiral through (4,0) with P 2 - The second iteration subdivides P\ into 
T 5 = Pi n Prer{Pi) = {(a;, 0) : 0 -^x — +X 2 } and Pi \ Pre-riPi) where X 2 > 0 
is the x-coordinate of the next point of intersection of the spiral with P\. This 
process continues indefinitely since the spiral intersects Pi in infinitely many 
points, and therefore the algorithm does not terminate. 

4 Subanalytic Sets and Stratifications 

In this section we describe some fundamental properties of subanalytic sets 
(sec [6,13,21] for more details). A differentiable manifold is real analytic (C“) 
if the transition maps between local coordinate charts are analytic functions on 
their domains (which are open subsets of K"). An embedded submanifold P of a 
manifold Af is a topological subspace of M together with a differentiable struc- 
ture such that the inclusion from S into M is & smooth immersion (i.c. has full 
rank at every point). A vector field F on the real analytic manifold M is analytic 
if its coordinates in any local chart are analytic. If F is an analytic vector field 
then any integral curve of F is analytic. 

Let M and N be real analytic manifolds and let C^{M,N) denote the set 
of analytic functions from M into A". If / G C^{M, N) we say / is of class C“. 
Given an analytic manifold U, we denote by i7(C'“(17, K)) the Boolean algebra 
generated by the sets of the form {x : /(x) = 0 } or {x : /(x) > 0 }, where 
/ G C‘^([/,K). 

Definition 2. Let M be a real analytic manifold. A subset A of M is semi- 
analytic in M if for every p G M , there is an open neighborhood U of p in 
M such that P n A G A(C'“(P,M)). If A M is semianalytic in M we write 
A G SMAN(M). 
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Definition 3. Let M be a real analytic manifold. Define SBANrc(M) and 
SBAN(M) by 

1. A E SBANrc(M) if and only if there is (N,f,A*) such that N is a real 
analytic manifold, f G C'^[N, M), A* G SMAN(A^), A* is relatively compact 
and A = f{A*); 

2. A (E SBAN(M) if and only if A is the union of a locally finite collection 
of members of SBANrc(M). (A collection of sets C is locally finite if any 
compact set intersect only finitely many sets in C.) 

Wc say that A is subanalytic m M if A G SBAN(Af). It is easy to sec that 
A G SBANrc(Af) if and only if A is subanalytic in M and relatively compact. The 
following properties of subanalytic sets are easily derived from the definitions. 

1. SBAN(M) is closed under locally finite unions and intersections. 

2. If A G SBAN(M) and /: M (8^ N is of class and proper on A, the 
closure of A, then /(A) G SBAN(A^). (A function / is proper if f^^{K) is 
compact whenever AT is.) 

3. If A G SBAN(iV) and f: M ^ N is of class C“, then f-^{A) G SBAN(Af). 

The following two properties require more subtle proofs, but they give the 
first indication that this will be a suitable class of sets for our studies. 

4. If A G SBAN(M) then M \ A G SBAN(M). 

5. A subanalytic set has a locally finite number of connected components, each 
of which is subanalytic. 

Example 2. Points are subanalytic, and so is any locally finite union of points, 
for example Z" as subset of M". The empty set and M are both in SBAN(M). 
Let a, 6 G M, a < 6, then [a,b], [a, 6), {a,b] and (a, 6) are subanalytic in M. The 
open ball B{p, r) centered at p of radius r in M” is in SBAN(M"). 

Definition 4. Let M be a j'eal analytic manifold. An analytic stratification 
of M is a partition S of M with the following properties: 

1. each S G S is a connected, real analytic, embedded submanifold of M , 

2. S is locally finite, 

3. given two sets S, P E S , P S, such that S C\ P 0 then S ^ P and 
dim S < dim P. 

The sets in a stratification are called strata. 

The central result on stratifications for our analysis is the following. For a 
proof see [20]. 

Theorem 1. Let A be a locally finite family of nonempty subanalytic subsets 
of a real analytic manifold M . For each A G A, let F{A) be a finite set of real 
analytic vector fields on M . Then there exists a .subanalytic stratification S of 
M , compatible with A, and having the property that, whenever S G S, S ^ A, 
A G A, X G F{A), then either (i) F is everywhere tangent to S or (ii) F is 
nowhere tangent to S. (S is compatible with A is every set in A is a union of 
sets in S.) 
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Theorem 1 is illustrated by the following example. 

Example 3. Let F be the following analytic vector field on 

X = x‘^ + y‘^ 
y = 0 

which has an isolated equilibrium at the origin and points in the positive x- 
direction otherwise. Consider the following two subanalytic sets 

A'l = {(x, y) G I y ^0 and {x 0 1)^ + = 1} 

S 2 = {{x, y) G I y = 0 and 0 -^x ^2} 

shown in Figure 2. A subanalytic stratification of which is compatible with 
the sets Si, S 2 and the vector field F is also shown in Figure 2. It consists of 

— 0-dimensional strata 

= (0, 0), P 2 = (2, 0), and P 3 = (1, 1) 

— 1-dimensional strata 

= {(x, y) G I y = 0 and 0 < x < 2} 

—>€2 — {(x, y) G I y > 0 and 1 < x < 2 and (x ® 1)^ + y^ = 1} 

-^Cs = {(x, y) G I y > 0 and 0 < x < 1 and (x ® 1)^ -h y^ = 1} 

— 2-dimensional strata 

^£>1 = {(x, y) G I y > 0 and (x ® 1)^ + y^ < 1} 
-^D 2 ^R‘^\{Pl,P 2 ,P 3 ,Ci,C 2 ,C 3 ,Di} 

Notice that the vector field is tangent to Pi since it is an equilibrium as well as to 
Cl, Di and D 2 - The vector field is transverse to all the other strata. Moreover, 
Si ^ PiU P 2 U P 3 UC 2 U C 3 and A 2 = £1 U P 2 U Ci. 




Fig. 2. Subanalytic stratification example 

In view of the above properties we will restrict our study to hybrid systems 
for which the relevant sets are all relatively compact and subanalytic. 
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Assumption 1 : For each discrete state q the collection Aq consists of rela- 
tively compact subanalytic sets. In particular, we assume there exists a compact 
set K such that if A G Aq then A —>K . 

The partition Sq which serves as the initialization step of Algorithm 2 can 
now be assumed to be a subanalytic stratification compatible with Aq and the 
vector field F{q, -)4(as given by Theorem 1). 

The following proposition illustrates some of the good intersection properties 
that analytic curves have with subanalytic sets. The “finiteness” property indi- 
cated in the proposition makes it possible to define transitions between adjacent 
strata in a natural way. 

Proposition 1. Let I be an open interval, M a real analytic manifold and 
■y: I ^ M a real analytic function. Let S be a stratification of M by suban- 
alytic sets If [a, b] ^ I then there exists a finite partition {xi , . . . , of [a, b] 
with the property that for each i — 1 , . . . , n (8> 1 there exists a stratum Si E S 
such that 7((x',, x, +i))- 6V 

Proof. The family 2 = {7”^ (S') n [a, 6] : S G S} is a finite partition of [a, 6] by 
subanalytic sets. Each such set consists of a finite number of points and open 
intervals. Using all such points and the endpoints of such intervals gives the 
desired partition. 

The following example shows the type of pathological situations that can be 
encountered if the assumption on subanalyticity is even slightly relaxed. 

Example f. Consider the stratification of by the following five sets; 

51 

5 2 

Si 
Si 

Si 

Notice that S\, S 2 and S 3 form the graph of the function f{x) = xsin^ 
(f(0) — 0), while S4 and S5 denote the region above and the below the graph, 
respectively. Each set is a , embedded submanifold of and they clearly sat- 
isfy the condition on the dimension of the strata in the closure of other strata. 
Finally, consider the constant vector field F = -^. Then the integral curve 7 of 
F through (0,0) is the x-axis (parameterized by x itself). Therefore, the image 
by 7 of any interval containing 0 intersects both S 4 and ^5 an infinite number 
of times. This is reminiscent of the undesirable zeno property which allows an 
infinite number of switehes in finite time. 



= {( 0 , 0 )} 

— I (x,y): x > 0 A y — x sin — I 

I 

= < (x, y) : X < 0 A y — x sin — I 

I 

= I (x, y) : X 7^ 0 A y > x sin ^ | [J {(0, y) : y > 0} 



= < (x, y) : X 7 ^ 0 A y < x sin 






y < 0 } 
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Fig. 3. Infinite crossings on a compact interval 



Since the algorithm considers one discrete state at a time, we will simplify 
the notation by assuming that the discrete state q is fixed and drop it as a 
subscript. In particular wc will consider a vector field F and a stratification S 
of Xc by subanalytic sets as provided by Theorem 1. By Xc/ -^we will mean 
the partition of Xq induced by S. We will denote by jx the integral curve of F 
which passes through x at time 0, i.e. with 7a;(0) = x. 

Wc now proceed to formalize the notion of a discretization of the continuous 
transitions relative to a given partition S. We do this mainly it simplifies the 
arguments in the proof of the main theorem (Theorem 2). In addition it sup- 
ports the intuitive picture we have that a trajectory can be decomposed as a 
concatenation of pieces in each of the sets in S. 

Definition 5 (Transition relative to S: version 1). Given x,y £ Xc we 

say X ^ y iff there is t > 0 such that £xif) = V o,nd there exists S £ S such that 
7a; (s) G S for 0 < s < t and at least one of x, y is in S . 

To clarify this concept and to facilitate further discussions and proofs we 
introduce additional dehnitions. 

Definition 6. Given two subsets Si, S2 of Xc, and a real analytic curve 7 : 
I Xc where I is an open interval, we say that 7 leaves through S2 (or 

enters S2 from Si) if one of the following exiting conditions is satisfied: 

El there exist a,b £ I , a < b, such that 7(f) S Si for all t £ (a, b) and ^{b) £ S2 

E2 there exist a,b £ I , a < b, such that 7(a) G Si and 7(f) G S2 for all t £ (a, b). 

When X £ Si we say that 7^ leaves Si trough S2 if either El or E2 holds with 
a — Q. 

The following proposition is a simple application of Proposition 1 and shows 
that Definition 6 covers all possible “exiting” situations for strata of S. 
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Proposition 2 . Let S\ € S and 7 be as above. If there exists to,ti G I such 
that ''/{to) G Si and 7 (^ 1 ) 0 Si then there exists a stratum S 2 Si) such that 
either El or E2 holds. 

It is clear from Definition 6 that in case El, 5'2nS'i ^ 0. By the properties of 
stratifications, we conclude S 2 D and dimS '2 < dimS'i. Therefore, the flow 
exits the stratum though a stratum of lower dimension. Similarly in case E2, 
Si D S 2 and dim < dim S 2 and the flow enters S 2 from a stratum of lower 
dimension. The following proposition further clarifies the possible exit situations. 



Definition 7. We call a stratum S G S tangential if the vector field F is tangent 
to S at every point of S. We call a stratum transversal otherwise. 

Proposition 3. Let Si, S 2 he strata in S and 7 an integral curve of F which 
leaves Si through S' 2 . Then one (and only one) of the following holds: 

1. condition El holds, Si is a tangential stratum and S 2 is a transversal stra- 
tum. 

2. condition E2 holds. Si is a transversal stratum and S 2 is a tangential stra- 
tum. 

We can now give the alternative definition of relative transitions. 

Definition 8 (Transition relative to S: version 2). For each x G Xq let 

S[x) denote the unique stratum in S which contains x. Given x,y G Xc we say 
X ^ y iff "fx leaves S{x) through S{y). 

It is clear from Proposition 1 that x ^ y iff there exist xi, . . . , such that 
X xi ^ Xn y ■ We will denote the Pre operator associated to ^ by 

Pres. The above remark also implies that we can substitute Pres for Pre^ in 
Algorithm 2 in the sense that if the algorithm terminates using Pres then it 
also terminates when using Pre^-. 

As the stratification Theorem 1 shows, issues of transversality of trajectories 
can be analyzed within the context of subanalytic sets and analytic vector field- 
s. However, the study of continuous transitions requires that we investigate the 
global behavior of trajectories. In general, trajectories of analytic vector field- 
s (and much less their full flows) are not subanalytic. Identifying vector fields 
whose flows belong to a suitable class is the main obstacle in the study of bisim- 
ulations of hybrid systems. Recent developments in logic model theory provide 
some answers as well as suggest the proper context in which to carry on further 
studies. 

5 Model Theory 

Model theory studies structures through properties of their definable sets 
(see [14,23] for general background). The basic structures of interest for this 
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paper are that of the real numbers as a complete ordered field, symbolized by 
(K, +, ®, £, <, 0, 1), and its extensions. Every such structure L has an associ- 
ated language £ of formulas. The (first order) formulas over C are the well- 
formed logical expressions obtained by using logical connectives, quantihers 3 
V, real numbers as constants, the operations of additions and multiplication, 
and the relations < and = (quantification is allowed over variables). All for- 
mulas will be interpreted over the real numbers. A definable set in the lan- 
guage C (or of the structure L) is a subset of K” (for some n) of the form 
{(ai, . . . , a„) G M” : . . . , a„)}, where . . . , as'„) is a formula in £ and 

x\,. . . ,Xn are free (i.e. not quantified) variables in A function / is definable 
if its graph is a definable set. 

While many of the concepts here apply to more general structures, in all that 
follows we consider only structures over the real numbers. 

Definition 9. The theory of C is o-minimal (“order mmimal”) if every defin- 
able subset o/K is a finite union of points and intervals (possibly unbounded). 

Tarski [22] was interested in the extension of the theory of the real numbers 
by the exponential function, (M, -|-, ®, £, <, 0, 1, exp) (i.e., there is an additional 
symbol in the language for the exponential function). We denote this structure 
by Mexp- While such theory does not admit elimination of quantifiers, it was 
shown in [25] that such theory is model complete, which in turns implies that 
it is o-minimal. Another important extension is obtained as follows. Assume / 
is a real-analytic function in a neighborhood of the cube [(8)1, 1]" £ K". Let 
/: M” ^ K be the function defined by 



fix) 



f{x) if X e [(81, 1]" 
0 otherwise 



We call such functions restricted analytic functions. The structure Kexp,an = 
(K, +, (8, £, <, 0, 1, exp, {/}) is then an extension of Mexp where there is a symbol 
for each restricted analytic function. One reason this structure is relevant for 
this paper is that all relatively compact subanalytic sets are definable in Kexp,an- 
Moreover, if £ is a linear vector field in K" with real eigenvalues, then the 
trajectories of F are definable in Kexp.an- In [24], it was shown that Kexp.an is 
also o-minimal. Finally, there are a few consequences of o-minimality that are 
crucial for our results. We list them below under one proposition. The proofs are 
contained in the various references mentioned above. 



Proposition 4. Assume L is an o-minimal structure. Then 

1. Any definable set has a finite number of connected components, each of which 
is a definable set. 

2. If A is definable, then so is its (topological) closure. Moreover, dim£r(A) < 
dim A, where Fr{A) = A \ A is the frontier of A and the dimension of a 
set B £ K" is the maximum integer d for which there is an embedded 
manifold o/K" contained in B. 
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3. Given definable sets Ai, . . . , in K" (and for any integer p), there is a 
finite stratification o/K" compatible with {A\, . . . , A^}. In fact, for the 
structure Mexp, an the strata are definable (real) analytic manifolds. 

We are now ready to apply these results to prove that Algorithm 2 termi- 
nates for certain classes of planar systems. 

6 Finiteness Results 

In this section we use the model theoretic tools of Section 5 in order to obtain 
classes of systems for which the Bisimulation Algorithm of Section 3 terminates. 

Recall that given the family of sets A as in Assumption 1, and the vector field 
F we hrst obtain a stratiheation S compatible with A as given by Theorem 1. 
We will also assume that S is compatible with a compact subanalytic set K 
which contains all sets in A. We define Sk — {S & S : S D K A (which is 
therefore finite). 

Theorem 2. Let Xc = F be the linear vector field Ax and assume that the 
eigenvalues of A are real. Then the bisimulation algorithm for hybrid systems 
(Algorithm 2), initialized with Sk, terminates. 

Proof. We will consider the case when the origin is the only equilibrium of F. 
(The other cases require minor modifications.) We assume without loss of gen- 
erality that {(0,0)} G Sic- 

As indicated in Section 3 it suffices to study only the evolution of the con- 
tinuous variables and use Pres in Algorithm 2. To simplify notation we will 
simply refer to it as Pre. In order to show that the bisimulation algorithm ter- 
minates we will construct a finite refinement of Sk which is “invariant” under 
the Pre operation and which is a refinement of Xcj T at each step. 

For each stratum S G Sk with (0, 0) G A we consider the set 

5oo = (xG5:Vt/:0 7.(i)e-S) 

As mentioned earlier, since the eigenvalues of A are real, the flow of F , 
d?{x,t) = 7x(t) = e^^x is definable in K.exp,an (the entries in e^^ involve poly- 
nomials and real exponential functions). Therefore, the set Soo is definable. For 
each stratum T of dimension one with T C S, T A S, we consider the set 

T* = (x G T : 72 , leaves T through Sqc} 

The set T, is also definable in Kexp,an and therefore can be written as a finite, 
disjoint union of definable sets each of which is either a point or homeomorphic 
to an open interval. We may assume, by refining the original Sk if necessary 
that the finitely many points in the decomposition of T* are already strata of 

Sk- 

For each x G let T} denote the trajectory of F passing through x, that is 

As = {ixit) : t G K}. 
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For each stratum S € S and x G S, let r^{S) denote the connected component 
of /Ij n S' which contains x. It is clear, from the definition of Sqo, that if x G Sqo 
then rx{S) C S^o- From this it follows that if x G T and leaves T through S 
then 7 j; either leaves T though S^o or leaves T through S \ Soo ■ 

Let {pi}, . . . ,{pi} be all the 0-dimensional strata of Sk- Notice that for 
each i,j, if Fp^ H Fp. yt 0, then Fp,. = Fpy We will eliminate redundancies and 
assume that the Fp^ are pairwise disjoint. For each set S G Sk and each Fp^, 
the sets S H Fp^ and S\ yJiFp^ are definable in Kexp,an (Intuitively, these sets are 
partitions of S “in the direction of the flow of F"). By o-minimality, we get that 
each such set has a finite number of connected components. Let B denote the 
(finite) collection of all such connected components. The collection B is then a 
partition of K compatible with S (every set in 5 is a union of sets in B) . 

Claim: At each step of the bisimulation algorithm, B is compatible with 
M/C. 

The claim shows that B is finer than all partitions obtained at each step. 
Since B is finite, this proves that the algorithm terminates. 

To prove the claim we first show that if Bi G B for i = 1, . . . ,n then 

Pre{\JBi) — UPre{Bi) (4) 

We will call a set B G B tangential if B is contained in a tangential stratum of S 
(i.e. B is a connected component of either S C\ Fq ot S \ UTp. with S tangential). 
The set B will be called transversal otherwise. Notice that if B is tangential and 
X G B then Fp{S{x)) C B. 

Let X G Pre{Bi) for some i = 1, ... ,n and x ^ S,. Suppose £ S{x) 

for 0 C t < 5 and jx{^) ^ Bi (i.e. exit condition El). In particular, S'(x) is a 
tangential stratum. If 7 a,(t) 0 UB^ for t < 6, then x G Pre{UBi). If 'jxit) G UB^ 
for some t < 5, then for some j, Bj is tangential, so Fx{S{x)) C Bj and x G 
Pre{UB^). If, instead, 7x(i) G B^ for 0 < t < 5 (exit condition E2), then clearly 
X G Pre{UBi). 

Conversely, let x G Pre{UBi). If 7x(t) G *S'(x) for 0 C t < 6, 7x(^) G UBj, 
let io be such that 7x(<f) G Big. Then x G Pre{Bi„) C UPre{Bi). If, instead, 
lx{t) G UBi for 0 < t < 6, then there is a do > 0 and a Big which contains 7x(t) 
for 0 < f < do (here we used o-minimality again to conclude that Fx intersects 
each Bi in a finite disjoint union of points and arcs). Therefore, x G Pre(Big). 
This conclude the proof of (4). 

By construction, B is compatible with Sk ■ At each step of the bisimulation 
algorithm we need to show that if B = and B' = U^^B' with B,, B' G B 

then B n Pre{B') is again a finite union of sets in B. Based on (4) it will suffice 
to show that for B, B' G B, either B H Pre{B') — /} or B D Pre{B') — B. 

We consider several cases. The set B is of one of the two forms: (a) a con- 
nected component of B H Fp. , or (b) a connected component of S' \ UFp. . 

If S is 0-dimensional there is nothing to show because B contains a single 
point. 

If S is 1-dimensional and B is of type (a), then either S is transversal and 
B consists of a single point or S is tangential and so B = Fx{S) for any x G B. 
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The first case is again clear. In the second case, if there is x € B D Pre{B') then 
there exists 5 > 0 such that 7 ®(t) G S for Q C t < 5 and ^ But then 

for all y G 'jy leaves S through B' . So B — B^iS) C Pre{B'). 

If S is 1-dimensional and B is of type (b) then we again consider separately 
the cases when S is tangential and when S is transversal. In the first case we 
proceed as before. Assume now, that S is transversal. Notice that ii x £ B D 
Pre{B') then B^ intersects both B and B' . Therefore B' is also a connected 
component of S' \ UTp, (for some S'). By transversality, jx leaves S intro S' 
under exit condition E2 and so S' £ Fron{S') (= S'\S') and S' is 2-dimensional. 
By continuity of the flow of T, there is an open neighborhood N ol x such that 
for y £ N D B, 'jy leaves S through S'. Moreover, since there are finitely many 
Tp, we may assume (by taking N smaller) that 'jy leaves S through B' . We 
have then showed that the set E — {x £ B : jx leaves S through B'} is open 
in B. Suppose E ^ B. Then there is y G i? in the frontier of E. We can find a 
neighborhood IT of y such that IT H Tp, = 0 for all i. Since S' is open in 
and S is transversal, we can find a neighborhood ITq £ IT of y and e > 0 such 
that for z G ITq H S and 0 < t < £ we have 7x(t) G IT Gl S'. But then every such 
2 ; belongs to E. This contradicts the fact that y is a frontier point. Therefore, E 
is also closed in B and so it must equal B (since B is connected). We conclude 
in this case that B = B f) Pre(B'). 

There is only one case remaining: S of dimension 2 (and hence tangential). 
If B is of type (a) then Bx{S) — B and we are done as before. 

Assume then that B is a connected eomponent of S \ UTp.^ , B' a connected 
component of S' \ UBp., S' is transversal, and dim S' = 1. (The case with S' 
0-dimensional is excluded since in that case S' H Bp^ / 0 for some i.) 

Let X £ B n Pre(B') and assume there is y £ B\ Pre{B'). We want to show 
that this leads to a contradiction. Let a : [0, 1] ^ S be a curve connecting x 
to y. Let to be the smallest t £ [0, 1] such that 7a(t)(s) ^ B' for some s > 0. 
If 7a(to)(s) G S for all ,s > 0 then a{to) £ S^o- By the choice of to we in fact 
have a{to) G Bp^ for some po (see the initial subdivision caused by Soo)- But this 
contradicts the fact that B is of type (b). Assume then that Ja{to)i^) ^ 
some s > 0. For each t £ [0, to] let s(t) be the smallest s such that 7 a(t)(s) ^ S. 
For each t G [0, to] set p{t) = 7a(t) ('S)^))- There are two possibilities: either 
p(to) £ S' or p{to) £ S'\S. 

In the first case choose a local chart {N, p) centered at p{to) so that in ip- 
coordinates we have N f) S' = N H B' = {(x, 0)} and iV n 5' = {(.x, y) : y > 
0} (therefore F points into the lower half plane at every point of N f) B' . By 
continuity of the flow and transversality, we still have that 7a(t) crosses NOB' 
from the upper to the lower half plane for to < t < to + e. But this contradicts 
the choice of to- 

In the second case, we have p{to) £ Tgo some qq. But this contradicts the 
fact that B is of type (b). 

All this implies that every y in B must also be in Pre{B'). That is, B = 
B n Pre(B'). This concludes the proofs of the claim and the theorem. 
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As the proof above suggests the termination of the algorithm depends on the 
fact that the integral curves of the vector field intersects relatively compact sub- 
analytic sets in at most finitely many points. This allows us to get the following 
generalization. 

Theorem 3. If F is an analytic vector field in which admits an analytic 
family of first integrals, then the bisimulation algorithm terminates. (Here, by an 
analytic family of first integrals we mean a non-constant (real) analytic function 
f : ^ K such that for each trajectory j of F the function f{'j{t)) is constant.) 

Proof. Notice that each level curve of / is an analytic set and therefore its 
intersection with any relatively compact definable set (in Kexp.an) is definable 
in Mexp,an- The proof then follows the lines of the previous one but replacing 
the sets Fj,., with the corresponding level set of / (level sets of / are at most 
1-dimensional since / is not constant on any open set). 

Corollary 1. If F is a linear vector field m with purely im, aginary eigenval- 
ues and Sk is as in the theorem, then the bisimulation algorithm terminates. 

Proof. Unless A = 0, in which case the result is trivial, there exists an (invertible) 
matrix P such that llPxlp is constant along trajectories of F. 



Corollary 2. If F is an analytic Hamiltonian vector field in and Sk is as 
above, then the bisimulation algorithm terminates. 

Proof. The Hamiltonian is constant along the trajectories. 



Remark 1. As is clear from the proofs above, the key is that all the objects 
involved (the vector field F, the initial family of sets, the flow of F) be definable 
in some o-minimal extension of the field of real numbers. We presented above 
just two specific instances of such a situation which can be easily characterized. 
A more recent o-minimal extension of the reals, by so called Pfaffian functions, 
was found in [26]. 

7 Conclusions 

In this paper, we presented new classes of planar hybrid systems with finite bisim- 
ulations. This was achieved by combining the geometric framework of subanalytic 
sets with model theoretic concepts from mathematical logic. The mathematical 
tools used in this paper provide the natural platform for studying decidability 
of computational algorithms for hybrid systems. 

Issues for future research, include the extention of these results to M” as well 
as to hybrid systems whose relevant sets and flows are definable in o-minimal 
structures. In addition, the issue of decidability requires not only termination of 
the bisimulation algorithm but also constructive decision methods for each step. 
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Even though decision methods exist for (M, +, (8>, <, 0, 1) [22], it is not known 

if the theory of Kexp is decidable, although in [16] it was shown that it would be a 
consequence of Schanuel’s conjecture in number theory. The results we obtained 
in this paper suggest how to find some restricted classes of vector fields for which 
the algorithm is constructive. Indeed, if all the relevant sets are semialgebraic 
(for example if E is a Hamiltonian vector field on the plane with a polynomial 
Hamiltonian and the initial conditions, guards, etc., are semialgebraic), then 
they are definable in (M, +, (g), £, <, 0, 1). Such a decidability result is obtained 
in [9]. 
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Abstract. This paper discusses recent results on multiple linear agent 
control for systems satisfying a bounded amplitude performance con- 
straint. The plant is assumed to be a linear parameter varying (LPV) 
system scheduled along a nominal parameter trajectory; in this respect, 
the control problem represents a plant operating between a prespecihed 
set of operating conditions. Linear controllers arc designed at setpoints a- 
long this scheduling trajectory to satsify bounded amplitude performance 
cosntraints. This paper discusses an approach to analyze the switched 
system behavior under practical assumptions on the structure of the 
switching rule. The approach combines the scheduling parameter with 
LPV system properties to derive bounds on the switching behavior of 
the system. These estimates are then used to construct a logical model 
of the switched system behavior in the form of a timed automaton. In 
this respect, this paper presents a way of extracting logical models of 
continuous time system behavior. 



1 Introduction 

Complex systems are often built up through the aggregation of simpler subsys- 
tems. A manufacturing system, for example, may consist of several robots whose 
individual actions must be coordinated in a way that prevents deadlock and en- 
sures the satisfactory completion of a specified task. Methods for the systematic 
design of such systems have only recently emerged. One of the major methods 
uses automatic verification methods applied to a timed transition system known 
as the timed automaton [1]. These verification methods determine whether or 
not the timed automaton satisfies a behavioural specification framed as a for- 
mula in the timed computational tree logic (TCTL) [2]. The verification method 
is based on real-time extensions of symbolic model checking methods [9] . 

An important first step in the analysis of such systems involves the extraction 
of timed automaton representations for the continuous-time system comprising 
the hybrid dynamical system. For example, in the robotic example outlined 
above, each robot can be characterized by a set of ordinary differential equa- 
tions. Because the robot functions over a very large operating range, however, it 
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is often necessary to use multiple models of the robot’s behaviour and to devise 
an appropriate switching rule for deciding which model and/or controller of the 
system to use. This approach leads in a natural manner to viewing each subsys- 
tem in the hybrid system as a switched dynamical system whose timed behaviour 
can be readily modeled using a timed automaton. Once such an automaton has 
been extracted, it can be used in conjunction with existing automatic verification 
tools to analyze the entire system’s behaviour. 

This paper presents a method for extracting a timed automaton model of this 
switched dynamical system. The remainder of this paper is organized as follows. 
Section 2 defines notation. Section 3 introduces the basic switching control sys- 
tem to be considered in this paper. Section 4 summarizes some recent results in 
bounded amplitude control of such switched systems. Section 5 discusses the use 
of these results in the extraction of timed automaton models. Section 6 discusses 
the application of these results for the bounded-amplitude stability of infinitely 
switched model reference systems. Section 7 summarizes the current status and 
future directions of this work. 

2 Mathematical Preliminaries: 

Definition 1. For a finite constant T > 0, the finite-horizon infinity norm of a 
signal f : 9R+ ^ 3?" is defined as 

ll/lloo,[0,T] sup ||/(t)| 
te[o,T] 

where || ■ || denotes the Euclidean I 2 vector norm. The linear space £J^[0,T] is 
defined by 

CU0,T] := {/ : K+ ^ 3?" |||/||oo.[0.T] < ^} 

The subset {/ : 3?+ ^ 3?" | ||./’||oo,[ 0 .t] < l} C £^^^[0, T] is denoted BC^[0,T]. 
The infinite-horizon infinity norm of a signal f : 3?+ ^ 3f" is defined as 

ll/lloo.[0,cx)) •= sup ||/(t)|| 
tE[0,oo) 

where || • || denotes the Euclidean I 2 vector norm. The linear space £^[0,cxd) is 
defined by 

£^[0, 00 ) {/ : 3?+ ^ 3f" | ||/||oo.[0,oc) < 00 } 

The subset {/ : 3?+ ^ 3?" | ||/||oo,[0.oo) — 1 } ^ '^ooPi denoted S£^[0, 00 ). 

The spaces £^[0, 00 ) and B£^[0, 00 ) will often be denoted, respectively, £^ and 
BCl,. 

Definition 2 (Parameter Variation Set). Given a compact subset 0 c 3J% 
the parameter variation set denotes the set of all continuous functions map- 
ping 3?+ into O. For a finite T > 0 and a compact subset 0 C 0, the set £©[0, T] 
denotes the set of all continuous functions 6 G which map [0,T'] into 0. 
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Definition 3 (Linear Parameter Varying (LPV) System). Given a com- 
pact set 0 C 3?®, and continuous functions A : ^ 3?"^", i? : 3?® ^ ^nxn„ ^ 

0:3?®^ 3?"*^”, and D : 3ft® ^ S)fjn*xn„^ ^th linear parameter varying 

(LPV) system is a dynamical system whose dynamics evolve as 



±{t) 




-A{e{t)) B[e{t)Y 


x{t) 


z{t) _ 




c{e{t)) D{e{t)) _ 


w{t)_ 



where 6 £ Tq. The LPV system of definition 3 will be denoted by S{0, A, B, C, D). 



Finally, a positive definite matrix P £ 3?”^” which satisfies 

A'P + PA + aiP + —PB'BP <Q (2) 

tt2 

for A £ 3R"^", B £ 3?"“^", and scalars ai and 02 , will be denoted 

P £ FeasRic [A. B, ai, 02 ) 

3 Switched Multiple Agent Systems 

In this section we model the plant as a model reference system. We’re interested 
in the design and characterization of switched linear controllers which satisfy 
bounded amplitude performance measures. 

Recent work in switched agent control primarily concerns stabilization of 
linear, uncertain plants. In [10], linear control agents are employed and switched 
into feedback according to a prediction of controller performance. That work 
reveals insight into the behaviour of switching systems (£2 performance) and 
introduces the concept of dwell time, the time that a given control agent must 
remain in feedback with the plant between switches. Nonlinear control agents 
are used for bounded amplitude performance in [13]. That work also explores 
the effects of switching transients and demonstrates that bounds on switching 
transients can be derived in certain cases. The methods presented in [3] provide 
sufficient conditions for the bounded amplitude performance of switched linear 
parameter varying systems with a hnite number of switches. The principle results 
from this paper provide important insight into methods for estimating switching 
times and hence can be used to extracted a timed automaton abstraction of the 
continuous-state plant. 

We assume that the system to be controlled has a differential equation Xp = 
fp{xp,u). The desired control task is to move the plant state Xp along a known 
reference trajectory which is the solution to the Xm ~ fm{xm)- We will use a 
multiple agent controller of the form 

u = K(m, X, t) 

= T>{x, u, t) 



rn 



( 3 ) 

( 4 ) 
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where K{m, x, t) is assumed to be an LTI system and T>{x, u, t) is the switching 
logic used to select which controller will be placed into feedback. The objective 
is to design the family of controllers K(m, x, t) and the switching logic I?(x, u, t) 
such that the amplitude of a performance variable z = r{x,u,t) is bounded 
above by a constant 7 . In other words, we design to satisfy the following con- 
straint, 

sup ||T(x, u, t) II < 7 (5) 

ie[o, 7 '] 

One method for solving this problem is to use recently developed methods 
in the synthesis of bounded amplitude linear parameter varying control systems 
[4] [3]. In this method, we linearize the plant about its reference model and then 
convert it to a linear parameter varying system. The linearization has the form 

X = A^{Xm,X,u)x + Bux{Xm,X,u)u + Bj^x{Xm,X,u) ( 6 ) 



where 



Ax{Xm,X,u) 

Bux {Xra ) X^ u) 
^wx{,Xra^ X-, u) 



A. 

dxj 



'fp{Xrm 0 ) 



A 

du’’ 

fp^Xp^U^ 



fp{Xmt 0) 



d 



d 



fp{Xrm^^X ^^/p(x^, 0 )u 



( 7 ) 

( 8 ) 



In this method, once the plant has been linearized, the nonlinear error term, 
B-ujx, is added to the linear model as a disturbance input vector. 

The next step is to transform the linearized error system into an LPV system. 
By grouping the (xm, x, u)-dependent terms in the elements of A^, B^x, and Byjx 
into real parameters 0 i, 02 , ■ ' ' the linearized error model may be written as 
an LPV system. This grouping forms a map S : 3?" x 3^1" x 3?"“ ^ 3?® which 
is called the parameter mapping. For piecewise continuous bounded trajectories 
Xm{t), x{t), and u{t) the parameter mapping S produces a piecewise continuous 
trajectory 6{t) — S{xm{t),x{t).u{t)) which is called the parameter trajectory. 
The nominal parameter trajectory 9nom{l) •= 5'(xm(t), 0, 0). The nominal pa- 
rameter trajectory clearly represents the parameter trajectory associated with 
the reference model state trajectory. Deviations from the nominal parameter 
trajectory may thus be treated as perturbations to the nominal system. To help 
quantify the size of these deviations, assume that S' is a continuous mapping. 
In particular, assume that S is bounded so that there exists a known constant 
kzi > 0 such that for any Xm £ 3R", xi,X 2 £ 3?", and u\,U 2 £ 3?"“ 



Si(x„t,Xl,Ui) - Sj(Xm,X2,U2)| < ||zi - Z2II ( 9 ) 

where || • || denotes the Euclidean vector norm, Si{-, ■, •) is the ith element of the 
parameter vector and Zj = C\Xj + Di 2 'Uj for j = 1 , 2 . 
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The LPV error system can now be formally defined. Given performance 
weights Cl and D \2 and the intial assumptions listed above, let 0 C 3?^ be a 
compact parameter set such that 0nom{t) G and S is the parameter mapping. 
Consider the linearized error system given in equation 6 and let zl : 3?'® ^ 

: K® ^ and : 3R® ^ be defined so that Ax = A o S, 

Bux = Bu ° S, and B^x = B^, ° S. The LPV error system is the dynamical 
system 



x{t) = A{6{t))x{t) + Byj{6{t))w{t) + Bu{9{t))u{t) (10) 

z{t) = Cix{t) + Di 2 u{t) (11) 

y{t) = x{t) (12) 

for 9 — S{xm,x,u) G Tq. where 

B{9{t))= \B^{9[t))B^{9{t))\, 



C{9{t)) 



Cl 

I 



and D{9{t)) 



0 D\2 
0 0 



This system will often be denoted S{0, A, B, C, D). The exogenous input w{t) = 
1 is called the fictitious disturbance. 

Specific parts of the switched LPV error system are now described. Control 
Agents: Consider the LPV error system V(0, A, B, C, D). Consider a sequence 
of times, {C, t 2 , ■ ■ ■ , } and let T/c — {1, . . . , M}. The parameter vectors ob- 

tained by sampling the nominal parameter trajectory at times tj for i G T/c form 
a finite collection of design points. In other words. 



Design Points = {0:0 = 9nom{U)A G '^k} (13) 

The ith design point will be denoted as 9nlm- With each z G I^:, associate a 
control agent designed for the LPV error system when the parameter is fixed at 
the design point 0nim- The zth control agent will be represented by the matrix 
A(*) g so that, as assumed in section earlier, the control agents are all 

linear state feedback controllers. The collection of control agents will be denoted 
as K.. 

Switching Sets: Switching between the different control agents in /C will be 
controlled by the parameter vector, 0. In particular, associate with each element 
of A3 a compact subset of the parameter set 0. This set will be called the switching 
set] the zth switching set associated with control agent will be denoted as 
0i C 0. While this set can be chosen in many ways, attention in this paper is 
confined to switching sets of the form 



0r ■■= < 9 



sup 

j 






q(0 

^no7n,j 






out 



(14) 



(i) 

where 9j and j denote the jth components of the parameter vectors 0 and 
(i) 

9nom, respectively, and 9 out is a parameter quantifying the size of the switching 
set. 9 out will also be referred to as the switching parameter. 
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Nearest-Neighbor Switching Rule: For a given collection of control a- 
gents K. with associated switching sets, there are a variety of switching rules 
which can be invoked. In this paper attention is focused on a nearest neighbor 
switching rule. Given the switching parameter a collection of control a- 

gents /C, and a collection of parameter sets C = {0^} as defined in equation 
14, suppose that control agent is in the feedback loop at time to and as- 
sume that 0(to) £ 0i- Then the control agent will remain in the feed- 

back loop until the earliest time tg when the parameter trajectory 9{t) satisfies 

sup^ 9j{ts) — dnlm j ~ '^out- At time tg, the control agent is then switched 

into the feedback loop where m — argmin^gx^ ~ 9nom ■ A parameter tra- 

jectory will be said to be legal if and only if 6{t) G Uigj^0i for all t and 9{t) 
is continuous except possibly at times tg when 9{tg) lies on the boundary of a 
switching set 0. In particular, a legal parameter trajectory is denoted 0 G Tc- 
^From the preceding discussion it is clear a given control agent, is 

switched out of the feedback loop when the parameter 9{t) leaves the switching 
set 0i- In order to guarantee performance properties, it would be advantageous 
if the parameter 9(t ^ ) immediately after the switch will also be in the switching 
set 0j. To help guarantee this property, it will be assumed that for all t there 

exists I G Ik such that 9nom,i{t) - 9^1^^^ < -dg^^ where < dout ~ for 

s = 1, . . . , s. In other words, this is an assumption that the reference trajectory 
has been sampled “adequately” . 



4 Switching Results 

This section summarizes results which we have proven in [3]. Only the results 
are stated here. 

Proposition 1. (LPV Switching Lemma) Consider any finite constants r G 
(0,1] and 7 > 0, compact sets 0i,02 C 0, continuous m,atrix mappings Ai : 
sjjs ^ SR”X" and Bi : ^ j — 2 and constant matrices 0, G 

for i = 1,2. Let C = {0i, 02}- 

Suppose there exist constants a > 0, /3 > 0, and p > 0 and positive definite 
matrices Pi and P 2 such that 



rP2 < Pi (15) 

7 "^! > C[Ci (16) 

7" ^2 > 02^2 (17) 

Pi G FeasRic (^Ai(6l),Ri(6»),2/3-f -,q) V0 g 0i (18) 

P 2 G FcasRic{A2{9),B2{9),p,p) V0 G 02 (19) 
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Let w, X, and z be the input, state, and output, respectively, of the dynamical 



system 





/ 


Ai(0(t)) Br{9{t)) 


x{t) 


x{t) 


— > 


01 0 


w{t)_ 


z{t) _ 


— \ 


A 2 ( 0 (t)) B2{9[t)) 


x{t) 




>. 


02 0 


w{t)_ 



(20) 



where tg > 0, T £ {ts,oo), Q £ Tq (i-v. 0(t) is legal) If x'{0)Pix{0) < 1 and 
w £ BCfff , then for any switching time satisfying 



Is 1(1 ‘ — 



2^5 



(21) 



with parameter trajectory 6{f) £ Tq satisfying 0(0) G 0\ and 9{tf) £ 02, 



p||oo.[0,T] 



< 



The LPV switching lemma states three sufficient conditions for bounded 
amplitude performance of switch LPV error systems. 

1. The first condition is that the initial error state, x(0), lie in the ellipsoid 
{x\x' P\x < 1}. From theorem 1 of [4], equations 16 and 18 are sufficient 
to guarantee that this ellipsoid is invariant for times prior to tg and that 
any point x £ {x\x' P\x < 1} will also satisfy HCiiH < 7 . Thus, the first 
condition guarantees performance on the interval [ 0 , tg] and is sufficient to 
ensure performance over the interval [0, T] if no switch were to occur. 

2. The second condition is that the parameter trajectories must evolve over the 
switching sets 0i and 02 so that 9{t) £ Tq with the added restriction that 
0(0) e 01 and 9{tf) £ 02- This added restriction guarantees that while the 
system dynamics correspond to {Ai, Bi,Ci), i = 1 , 2 , the parameter trajec- 
tory lies in the set 0^; this condition ensures that the Riccati inequalities of 
the lemma are valid for the system. The condition of the lemma states that 
immediately after the discontinuity, the parameter trajectory must satisfy 
0 (t+) e 02 as shown. 

3. The final condition is that the switching time must satisfy a dwell-time re- 

quirement, tg > trf. This guarantees that the state error has had sufficient 
time to decay so that any transient associated with the switch will not violate 
performance constraints. At the time of the switch, the invariant ellipsoid 
associated with the system dynamics switches to {x\x' P 2 X < 1}. The dwell- 
time condition guarantees that at time tg, the state x{tg) lies on the interior 
of the new invariant by ensuring that the state has had sufficient time to 
decay into the ellipsoid {x\x' Pix < r} C {x\x' P 2 X < 1}. This decay time is 
characterized by the constant /3 which parameterizes a bounding exponential. 
The dwell-time is computed by determining when this bounding exponential 
satisfies = r. 
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Lemma 1. Given performance level 7 > 0 and LPV error system, A, B, C, 
D) suppose that the control input to the system is given by u(t) — Kxff) where K 
is a constant gain matrix. Let 6 be a point on the nominal parameter trajectory 
and let O be any compact subset of G containing 0. Suppose that there exists a 
positive definite matrix P and constants a > Q, (3 > 0, and 0 < r < 1 such that 

PP^iCi+DuKYiCi + DuK) ( 22 ) 

7 

P e FeasRic(A(0) + S„(6»)KB„(0),2/?+ V6» G 0 (23) 

For any T > Q, if x'{{))Px{{}) < 1 and w G then any parameter 

trajectory 6 G Pe[0, T] must also satisfy 

\0i{t) - 6'„om.i(t)| < fczj7max{Vr,e“^‘} 

for i = 1,2, . . . , s and for all t G [0, T] (24) 



Lemma 1 is important because it implies that if the bound on the parameter 
trajectory is small enough, then the nominal parameter trajectory represents a 
reasonable approximation to the true parameter trajectory. 

Corollary 1. Suppose that the conditions of lemma 1 are satisfied with adequate 
sam,pling and let T G [0,oc). Let 0 C 0 be a switching set. //x'(0)Px(0) < 1, 
w G BCfff[Q,T] and a parameter trajectory 0 G J-q[Q,T] satisfies 

sup \9t{T) - 9t \ = douL (25) 

!<*<s 

at time T , then the nominal parameter trajectory at time T satisfies 
dout-kzamax{^/f,e~^^} < \9nom,i{T)-0i\ < dout+kzi'ymax{^/f,e~^'^} (26) 
for i — 1, 2, . . . , s. 

Corollary 1 is useful because it implies that events in the parameter space 
(e.g. the parameter trajectory crossing the boundary of a switching set) can 
be predicted with the nominal parameter trajectory. The time at which the 
parameter trajectory may intersect the boundary of 0 may be approximated by 
the times at which the nominal parameter trajectory evolves over points near 
the boundary of 0. 



5 Automaton Extraction 

When combined with a specific switching rule such as the nearest neighbor rule, 
the nominal parameter trajectory can be used to estimate switching times and 
the results of possible switches. 
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Proposition 2. Given a performance level 7 > 0 and an LPV error system, 
S{0, A. B.C, D), let 1C be a set of control agents which forms a control policy 
with the nearest neighbor switching rule under adequate sampling. Suppose that 
at time to, the control input to the system is given by u{to) = K^^^x{to) where 
KC) 

G fC is a constant gain matrix. Let Oi be a switching set and suppose there 
exist positive definite matrix P and constant a > 0, /3 > 0, and 0 < r < 1 such 
that 

P>\{Ci + Di2K^‘^Y{Ci + 

7 

P G FeasRic (^A(6i) + Bu{0)K^’'\ By,[e),2f3 + V6» G Gi 

Define the sets 

0m ■■= [0 

and 



where 

kmaxAt) — 

If x'{to)Px{to) < 1, w G BCoo, and a parameter trajectory 0 is generated 
by the nearest neighor switching rule under adequate sampling, then the switch 
time, tg, between the Ith and mth systems satisfies tg G 



for i — 1, . . . , s and some 0 G and 



'd out blmax.iif) |^nom,i(^) d, 

for some 1 < i < s 



(0 



^ ‘dout ^ 



0e0,\\0-0tl\\ < \ A-0iYLlm,^eI^,qfm] 



(27) 

(28) 



Proposition 2 is important because it provides a way of estimating the time 
interval over which switches between agents will occur. There are two primary 
components to the construction of a set 

Switching Destinations The set 0„i represents the set of all parameter vec- 
tors 0 £ 0 which satisfy 



m = arg mm 
j€Iic 



0 - diA 



if the parameter trajectory at time tg lies in 0^, a switch to control agent m 
will take place according to the nearest neighbor switching rule. The sets 0m 
may be represented by a set of affine inequality constraints on the parameter 
vector 0. Note that these constraints may be computed in an off-line fashion 
with knowledge of the nominal parameter trajectory and the design points. 
Since the true parameter trajectory can only be estimated by the nominal 
parameter trajectory (as a consequence of lemma 1 ), the times for which 
d{t) ^ 6^771 CB-ii only loG cstiiiicitccl. Let tlic set 

■■= {0 I \0nom,i{t) - < kmax.i{t), for all i = 1, . . . , S } 
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For each time t, the set 0t is a hyperrectangle centered at 0nom{t) which 
contains the true parameter trajectory 0(t). The first requirement for a time 
t to belong to T*-*’"*^ is for the intersection 0 ^ 0 0 ^ to be nonempty, which 
is a relatively simple convex feasibility problem. 

Switching Times As a consequence of corollary 1, the times for which the 
nominal parameter trajectory satisfies 



'0out kjyiax,i{t) ^ \0noTn,i{ts) ^nom.i\ — '^out T 



for some i = bound the times for which the true parameter tra- 

jectory may intersect the boundary of the parameter set 0;. This condition 
represents an added restriction for a time t to belong to Note that 

this requirement may also be expressed with a set of affine inequality con- 
straints on the parameter vector 0 which may be computed in an off-line 
fashion. 

Given the preceding descriptions, the set T*-*’"*^ is constructed by finding all 
times for which the nominal parameter trajectory lies near the switching surface 
and for which the corresponding set Ot has a nonempty intersection with 0 ^. 
Theoretically, constructing the sets requires checking the feasibility of 

convex constraints for points along the nominal parameter trajectory at every 
time, t. Practically, constructing the sets requires checking for feasibility 

at a large number of points. This can be done efficiently using a binary search 
of the reference trajectory. 

We can now use the above results to extract a timed automaton. Control 
agent gains, were synthesized for the biased systems using the techniques 

presented in [4] . A MATL AB program was written to implement the conditions 
associated with propositions 2. First, the switch-time sets T*-*’*”) were computed 
according to the conditions of proposition 2. The nominal parameter trajectory 
was searched to determine possible switching times and the resulting switches. 
The result of the search is a tree depicting all possible switches and results for 
a fixed performance level 7 and switching parameter dout- 



[ 0 . 2701 , [ 0 . 3006 , [ 0 . 3518 , [ 0 . 6650 , 

0 . 2705 ] 0 . 3010 ] 0 . 3525 ] 0 . 6655 ] 







Fig. 1. Performance Validation Tree: d„ut — 1-5, 7 = 0.17 



Using the switching times, computed for the controlled system (as 

depicted in figure 1 ), one may construct a timed automaton [ 1 ] to represent the 
behaviour of the switched system. The timed automaton corresponding to the 
tree in figure 1 is shown in figure 2 ; it represents an abstraction of of the multiple 
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agent controlled system obtainable with estimated of switching times, the sets 
qJ? lemma 2. 

t<-0 



t e[0.0508,0.0519] 



Fig. 2. Timed Automaton for Multiple Agent System: 'dout — 1-5, 7 = 0.17. 



6 Bounded Amplitude Stability for Infinitely Switched 
Systems 

In the presence of bounded disturbances, it is important that we ensure the out- 
put of the system is also bounded. We therefore present the notion of bounded- 
amplitude stability (b.a.s.). This is essentially a requirement for input-output 
stability. 

Definition 4. Let j > 0 he a finite real number. Consider a system H described 
by the differential equations 

X = f{x, w) (29) 

z = h{x, w) (30) 

where f : ^ 3R" and h : ^ sr"z piecewise continuous bounded 

mappings. H is said to be bounded amplitude stable (b.a.s.) if and only if for any 
^ ^ ^^); II 1 1 CX5, [0,oo) 1 1 ^ 1 1 oo, [0,oo) — 7' 

We modify the control agents presented in the preceding section so they 
have an integrator in their forward loop. The introduction of the integrator 
ensures that the parameter trajectory, 9{t), will be continuous, so that the legal 
parameter switching lemma is no longer needed in the LPV switching lemma. 
The ith control agent will therefore be represented by the system 

u = V 

V = K^'^x + 




(31) 

(32) 
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where k[^'^ and are constant gain matrices of appropriate dimensions. The 
collectionof control agents will be denoted as K,. 

For analysis and synthesis purposes, the integrator in the control agent will 
be incorporated into the plant. The modified LPV error system is therefore given 
by 

x{t) = A{6{t))x{t) + B^{0{t))w{t) + Byv{t) (33) 

z(t) = Cx[t) (34) 



where 



mt)) 

Bum) 



'A{9{t)) B^{0{t)Y 
0 0 

\BU0{t)Y ^ ^ 



[Cl 




(35) 

(36) 

(37) 



and X — [x' m']. Consistent with this, we denote the fact that the integrator 
control agent has a controller if'*-*) = ^ 

We can now compute controllers for the given setpoints in the same way 
as was done above. Using the fact that the integral control agent generates a 
continuous parameter trajectory it can be readily shown that the legal jump 
condition is no longer needed. 

Note that in many systems, the reference trajectory 0m{t) may be a periodic 
signal. In this case, the continued operation of our system will result in an 
infinitely switched model reference control system. In this case, weTe interested 
in the stability of the infinitely switched system. 

Let denote the set T^^m) ^Q^iputed with kmax,iitY)- Since the dwell- 

times are fixed, so are the sets Using this information, a timed automaton 

may be formed representing the switching behaviour of the controlled system. 
One may then analyze the transitions of the automaton using the LPV switching 
lemma. If the transitions are separated by the appropriate dwell time, then 
the switched system is bounded amplitude stable or rather it exhibits bounded 
amplitude performance even for switching sequences of infinite duration. The 
following proposition states this result. 



Proposition 3. Given a performance level 7 > 0 and modified LPV error sys- 
tem A, B, C, 0) let K. be a set of control agents which form a control policy 
with the nearest neighbor switching rule under the assumption of adequate sam- 
pling. 

Suppose that for each I £ Tk, ther exists a positive definite matrix pb) 
constants oYY > oi^ > 0 such that 



p0) > 

7 

£ FeasRic (i(6») + B^K^‘\B^{0), 4*^) > V6> e 6>, 



(38) 

(39) 
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If, for all possible switching sequences, k I ^ m, there exists satisfying 

< r < l| 

and 

af - 

then the controlled system is b.a.s. 

In recent years there has been considerable interest in the stability of switched 
systems. Results presented in [6] and [7] indicate that the existence of a set 
of multiple Lyapunov-like functionals is sufficient for asymptotic stability in 
switched systems. Related work pertaining to linear time invariant switched sys- 
tems will be found in [11]. The determination of these Lyapunov-like functions, 
however, is non-trivial. For switched linear time invariant (LTI) systems whose 
switching sets form cones in the state space, linear matrix inequalities (LMI) 
[5] were simulatenously suggested in [12] and [8] as a means of computing such 
Lyapunov-like functions. It should be noted that the conditions obtained above 
for bounded-amplitude stability are similar to LMI conditions which have been 
previously reported. In our case, however, we solve a set of LMI’s associated 
with ensuring bounded amplitude stability of the switched system, rather than 
asymptotic stability. 

7 Summary 

This paper has outlined a method for the extraction of timed automatons used 
in the symbolic model checking approach to hybrid system verification. While 
work on automaton extraction for switched LTI controllers is well understood. It 
is obvious that nonlinear controllers may be more useful in certain applications. 
Our current research efforts are focusing on the design of gain-scheduled output 
feedback controllers. This work is near completion and initial results will be 
reported at the HS’97 workshop. 
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Abstract. We describe model semantics and develop a simulation algo- 
rithm for characterizing a class of dynamic physical systems operating in 
the so-called sliding regimes. Complex continuous system behavior com- 
bines effects that occur at multiple temporal and spatial scales. Behavior 
generation is simplified by creating system models that employ time s- 
cale and parameter abstraction techniques. The resultant hybrid systems 
exhibit discrete and continuous behaviors, which manifest as piecewise 
continuous behaviors interspersed with discontinuous changes between 
the continuous operating modes. Mode transitions are induced by inter- 
nal state changes and external control signals. Sometimes hybrid systems 
exhibit chattering behaviors at the discontinuous transition boundaries. 
This presents computational challenges to conventional numerical sim- 
ulation methods. We develop an efficient, adaptive algorithm for sim- 
ulating this class of systems, based on a careful analysis of the model 
semantics at the discontinuous boundaries. Simulation results show that 
the algorithm is more efficient and accurate for sliding mode systems 
than conventional integration methods. 



1 Introduction 

Complex physical systems exhibit behaviors that occur at multiple temporal and 
spatial scales. To simplify behavior analysis, system models incorporate abstrac- 
tions that (i) ignore small parasitic effects, and (ii) compress fast behaviors to 
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occur at a point in time. The resultant model behaviors are piecewise contin- 
uous, i.e., they include modes of continuous operation with discrete transition 
between modes. Building these hybrid system models is a difficult task [10]. In 
previous work [8,15], we have developed a compositional modeling approach with 
local switching functions to develop hybrid models of physical systems. Dynamic 
mode switching is attained by a meta-level control model operating on top of 
the data flow model. The control model is usually a discrete-event system that 
composes together model fragments to define behaviors in different modes of 
operation. Examples of hybrid systems include traffic control systems, electric 
power circuits, reprographic machines, embedded manufacturing processes, and 
economic models. 

Building numerical simulators for these systems is often hampered by the fact 
that steep gradients occur because of fast nonlinear phenomena. Hybrid models 
abstract away fast nonlinear effects by invoking discrete mode change functions 
when system state variables reach or exceed threshold values. The threshold 
functions can be looked upon as switching surfaces in phase space along which 
discontinuous changes in the system may occur. ^ Large errors may be avoided 
in traditional integration schemes like the Runge-Kutta method by using very 
small time steps around points of discontinuity where steep gradients may occur, 
at the expenses of incurring significant computational overhead. 

The ability to accurately analyze hybrid systems while retaining computa- 
tional efficiency requires: 

— well-defined semantics for defining model components and transition func- 
tions, and 

— simulation schemes that can seamlessly combine continuous behavior gener- 
ation methods with discrete mode switching schemes. 

This paper focuses on modeling semantics and simulation schemes for a class 
of hybrid systems operating in the so-called sliding regimes where the system 
chatters between two modes of operation. An example of such a system is the 
anti-lock braking system employed in automobiles. The switching surfaces in the 
physical system behavior description arise from modeling artifacts that abstract 
away the hysteresis effects of small, unmodeled parameters. We develop a simu- 
lation algorithm based on Filippov’s construction of equivalence in dynamics in 
sliding regimes [3]. This simulation algorithm is known to bo more efficient than 
conventional schemes that use a fixed step size in behavior generation. 

The sliding mode simulation algorithm is based on the observation that the e- 
quivalent dynamics on a sliding surface correspond to the limiting behavior when 
switching tends to be infinitely fast. In previous work [17], we have developed 
an adaptive algorithm that accurately follows a sliding trajectory and generates 
control signals at discrete times by exploiting the equivalence in control signals. 
In contrast, the algorithm described in this paper exploits equivalent dynam- 
ics for sliding mode systems and presents an alternate method for adaptively 
following trajectories at discontinuous boundaries. 

^ A phase space of a dynamic system is a multi-dimensional space defined by the 
independent state variables of the system. 
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2 Examples of Hybrid Systems 

A typical example of a hybrid system, shown in Fig. 1, is the evaporator vessel 
in the secondary cooling loop of a fast breeder nuclear reactor [7], In this system, 
a sodium pump maintains a sufficient flow of coolant in the loop. To keep the 
level of fluid in the evaporator vessel at or below a pre-specified maximum, 
an overflow mechanism is activated to drain excessive fluid once this level is 
attained. As more fluid comes into the evaporator, the flow through the overflow 
pipe builds up momentum. The evaporator behavior, described by the level of 
fluid in the evaporator, L, and the fluid momentum, p, is illustrated for a fixed 
inflow as a continuous phase space diagram, shown in Fig. 2(a) for a given inflow. 
At and around the overflow level, Lth, the continuous but nonlinear behavior is 
governed by adhesive forces in the fluid, the intake area of the overflow pipe, and 
the fluid surface area. Inspite of the nonlinearities the system attains a steady 
state behavior (see Fig. 2(a)). To simplify analysis, the detailed nonlinear effects 
can be abstracted away, and the overflow mechanism is modeled as having two 
distinct modes of operation; (i) when the fluid level is below the overflow level, 
and (ii) when the fluid level is above the overflow level. Behavior in each of 
the modes can be simplified to an almost linear trajectory (see Fig. 2(b)). This 
behavior abstraction produces simpler piecewise continuous behaviors with a 
discrete switch from one behavior to another at the point in time when L = Lth-, 
which defines the switching surface. We refer to this form of model abstraction 
as a parameter abstraction [8] . 




Fig. 1. A hydraulic system with two distinct modes of operation. 

Behavior can also be abstracted in time. Tim,e scale abstraction compresses 
the effects of fast change to occur at a point in time. The resulting behavior 
may become discontinuous [8]. An example is the bouncing ball shown in Fig. 3. 
A falling ball hits a floor with negative velocity, i.e., Vbaii < 0. The point of 
contact, where the vertical position of the ball x^aii = 0, defines the switching 
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Piecewise simpler model. 



surface. At this point, fast phenomena occur that transform the kinetic energy 
of the ball into stored elastic compression energy in the ball, and this is again 
turned back into kinetic energy as the ball decompresses, and the ball velocity 
Vbaii reverses. A phase space plot, Xbaii versus v^aiu of the continuous behavior 
is shown on the left of Fig. 4. The discontinuous change introduced by time 
scale abstraction is shown on the right in Fig. 4. The velocity of the ball Vbaii 
undergoes an instantaneous jump from ®Vbaii to Vbaii- This is shown by the 
double arrow heads in Fig. 4. 




Fig. 3. A bouncing ball. 



Introducing parameter and time scale abstractions into physical system mod- 
els results in piecewise continuous behaviors. Behavior switching occurs at points 
in time on switching surfaces, and this often leads to jumps in state variable val- 
ues. The mixture of continuous and discrete behaviors collectively defines the 
notion of hybrid models [4,8,10] of physical systems, represented by an integrat- 
ed formalism that combines continuous differential equation models with discrete 
switching functions implemented as Petri nets or finite state automata [1]. This 
paper develops a simulation algorithm for accurately approximating behaviors of 
a class of dynamic systems near switching boundaries that result from parameter 
abstractions. 
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Fig. 4. Phase space of a bouncing ball: (a) Continuous system, and (b) 
Instantaneous mode change. 
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Fig. 5. Types of phase space behaviors near a switching surface: (a) 
The vector field approaches the surface from one side and leaves from 
the other side; (b) Vector fields on both sides point away from the 
surface; (c) Vector fields on both sides point towards the surface. The 
figure shows these three configurations for two different combinations 
of tangential components of the vector fields. 
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At a discontinuous boundary when model switching occurs, a wide variety 
of behaviors may be observed. A classification presented in [16] and illustrated 
in Fig. 5, categorizes three types of behaviors. They are characterized by the 
directions of normal components of the vector fields, and fp in modes a 
and P, respectively. Discrete-time simulators are prone to make large errors 
at switching boundaries in situations shown on the left in Fig. 6. A way of 
minimizing these errors is to reduce the simulation time step in the vicinity of the 
switching boundary so the cross over point may be estimated more accurately. 
A discontinuity surface can be accurately located by successive halving of time 
steps, commonly referred to as the time-step halving method [11]. 




Fig. 6. Errors without time-step adjustment near discontinuities (left) 
and with time-step halving (right). 



Chattering occurs when field vectors in adjacent modes of operation are di- 
rected toward the switching surface (Fig. 7). In some cases, such as anti-lock 
braking systems [12,14], chattering may be an intentional effect. In sliding mode 
operation, the system switches between modes at a very fast rate on the switching 
surface, producing a fast chattering behavior. This causes the simulated aggre- 
gate behavior along the switching surface to progress slowly in time. Chattering 
limits the step sizes of numerical integration for accuracy reasons. Variable step- 
size methods are not applicable here since these methods rely on continuous 
properties of the system variables. To reduce the computational complexity due 
to the slow progression along the sliding surface, the system dynamics along the 
surface can be approximated by two methods: (i) equivalence in control [17], and 
(ii) equivalence in dynamics [3]. In either case, a larger step size can be employed 
by the integration scheme without introducing intolerable errors. 

3 Chattering in Physical Systems 

We study the physical origins of chattering using two characteristic multi-mode 
systems in order to derive a consistent model semantics for simulation: (i) the 
secondary cooling loop evaporator vessel, and (ii) a cam-followcr mechanism. 
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Fig. 7. A hybrid system may chatter. 

3.1 The Evaporator 

Consider the hydraulics of the evaporator vessel in Fig. 1. There is a constant 
inflow of liquid into the tank, fm, and an outflow, font, that depends on the 
pressure in the tank and the Bernoulli resistance, Rb. As discussed, the system 
can be modeled to operate in two distinct modes of operation: (i) mode a, where 
there is no overflow, and (ii) mode /3, the overflow mode. The overflow mechanism 
activates when the liquid level in the evaporator, L, exceeds the threshold value, 
Ltb- This causes a flow fsump through a narrow pipe with resistance, Rpipe, and 
inertia, I. When the overflow mechanism becomes active, the pipe inertia builds 
up flow momentum, p, till steady state when /i„ = font + fsump- 

p = (^5xiv±p 
p = (^Ex^pj^L 

L = ® + %" 

Suppose the system is in mode a initially. The inflow causes the tank to start 
filling, and this in turn causes an outflow through resistance Rb- The outflow 
through the narrow pipe is zero. If steady state is attained before overflow, the 
inflow, /,„, into the tank equals outflow, font, of fluid and the level of liquid in 
the tank L becomes steady. However, if L exceeds the threshold level, Lth, an 
immediate switch to mode (3 occurs (see Fig. 8). The second outflow path in this 
mode slows the pace at which the tank fills, and this continues till a new steady 
state level, sp, where total outflow equals the inflow. This new steady state liquid 
level is below what would have been attained had the overflow mechanism not 
been present. 

The behavior is more complicated when the level Lth is greater than the 
steady state level L attained in mode /3 (Fig. 9, the mode is marked in the 
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Fig. 8. After an initial transient stage, the evaporator level reaches 
steady state. 



bottom left corner). When the overflow is active in mode /?, the system moves 
towards a steady state with a lower level value. This can cause L < Lth, which 
turns off the overflow mechanism, and the system moves back to mode a. The 
grayed out area in Fig. 9 in each mode represents state veetor values that cause 
a transition to the other mode. The fields in the two modes are directed towards 
the switching border (Ttfi), and, therefore, independent of the initial conditions. 
As a consequence, the system can start chattering (see Fig. 10). 
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Fig. 9. Phase space of behavior in each mode. 



Analysis of the chattering behavior requires a physically consistent treatment 
of state evolution at Lih. Since the domains of the fields for each mode are 
mutually exclusive, their phase spaces can be combined. Fig. 11 depicts three 
scenarios by which Lth may be approached. In the scenario marked 1, the system 
approaches Lth with a field component in the direction in the active mode 
of operation (3. In scenario 2, the behavior path to Lth has a 0 component in the 
p direction in mode /3. In scenario 3, the field definitions in modes a and [3 have 
equal angles but opposite directions of approach toward Lth- The objective is to 
determine which one of these scenarios defines equilibrium when Lth is reached. 

To investigate physical behavior, we first observe that in reality the border 
between the modes of operation is not as crisp as modeled. Modeling abstractions 
disregard small parameters that are present and affect behavior at the boundary, 
Lth- For example, though small, forces at the rim of the overflow pipe require 
the fluid level in the tank to be somewhat higher than the rim in order for 
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Fig. 10. Chattering between modes with an active and inactive over- 
flow, Rb = 1, Rpipe = 0.5, 1 = 0.5, C = 15, /„ = 0.25, AT = 0.025. 





Fig. 11. Concatenation of pieces of phase spaces from modes a and (3. 
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liquid to start pouring in. Therefore, there is a finite time interval in which this 
excess level is drained, and the overflow mechanism is active. When the level falls 
below Lth the overflow mechanism turns off again. Other higher order physical 
phenomena, such as cohesive forces in the liquid, result in continuous as opposed 
to discrete switching behavior around Lth- 

Other physical effects also contribute to the hysteresis phenomena at the 
boundary between operational modes, and this can be used to derive correct 
model semantics for behavior generation at mode boundaries. Fig. 12 shows the 
effect of a □ e hysteresis band around Lth- Clearly, the system converges to a 
recurring point on Lth ® e and Lth + e and starts to oscillate between them. At 
lim^^o these recurring points coincide, and the resultant at the common point 
Lth can be computed from the limit values of the field in a and /? at this point. 
If e is small, the curvature of the field in j3 approaches a straight line. If the 
direction of the field in a is opposite to the field direction in j3 at the boundary 
Lth, this point is stable. Trajectory 3 illustrates such a stable point in Fig. 11. 




3.2 The Cam- Follower 

The cam-follower system in automobiles [5] translates rotational motion into a 
linear displacement to open and close valves in the engine cylinders (Fig. 13). 
A spring mechanism ensures contact between the rod and rotating cam but the 
high rotational velocities (up till several thousands of revolutions per minute) and 
wear in the spring may result in the rod bouncing on and off the cam, producing 
a collision phenomena similar to the bouncing ball, where mode switching results 
in jumps in the values of state variables (Fig. 4). Newton’s rule for collisions may 
be applied using a coefficient of restitution, e, to model loss of energy during 
collision (8> = ®^{vrod ® Veam)- Typically, e is a function of impact 

velocities [2]. If impact is less than a threshold value, Vrod ® Veam < '^’th, the 
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collision can be defined to be perfectly non-elastic (i.e., the rod and the cam 
move together). 




Fig. 13. A cam mechanism opens a valve. 



To study the behavior of the cam-follower mechanism in phase space, consider 
the valve and rod mechanism motion with the valve spring action and the rocker 
arm friction. 



/?: 



^rod — 9 ^ 0 ^'^rod 

^spring — ^'^rod 



( 3 ) 



The behavior of the valve spring and combined inertias, a second order system 




Fig. 14. Phase space of the cam-follower. 



with friction, is illustrated in phase space on the left of Fig. 14. The rod velocity 
oscillates between positive and negative with a decreasing amplitude. The cam 
mechanism forces the rod velocity to follow an ellipsoid path as shown in the 
right phase space diagram in Fig. 14. 

I ^rod — ^cam / a\ 

^ \ F — —V I 

spring — q ^roa 

When the rod is detached from the cam system, a collision can occur if the rod 
and cam positions are equal, and Vrod < Vcam- If the collision is perfectly non- 
clastic, the rod velocity instantaneously equals the cam velocity, as is indicated 
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by the grayed out areas of the phase space in Fig. 14. The rod disconnects from 
the cam if its deceleration is larger, and this corresponds to the steeper curve in 
the left half-plane (see Fig. 15). 




Fig. 15. The rod may disconnect, R — 10,m — 0.5, C — 0.02, AT — 0.01. 



Simulation runs using numerical approximations of the rod parameters in- 
dicate that the collision phenomena described above does occur (see Fig. 16), 
and the rod begins to exhibit a phenomena where it goes through a sequence of 
connects and disconnects, i.e., it begins chattering along the switching surface 
Fnormai = 0 and Vrod = Vcam as shown on the right in Fig. 16. Like the evapora- 
tor, chattering is an artifact of the simulation caused by parameter abstraction 
in the model. In reality, the rod’s elasticity and adhesive forces between the rod 
and cam surfaces generate higher order behaviors that ensure that the connec- 
tion remains for a short while before the two disconnect. In the limit, as the 
values of these parameters tend to 0, the system exhibits sliding behavior along 
the Vrod = Veam surface. 

3.3 Model Semantics 

When a system operates in the sliding regime, system dynamics in the vicinity 
of the sliding surface does not appear to be continuous. We adopt the notion of 
equivalence in dynamics for the sliding regime developed by Filippov [3,14], and 
compare it against an alternative approach, equivalence in control [17]. 



Equivalence in Dynamics Consider the switching surface as an infinitesimal 
band rather than a crisp border. Equivalent dynamics on the surface is defined as 
the behavior in the limit as the width of the band tends to zero. This construction 
preserves the physical meaning of the dynamics at the discontinuous boundaries. 
Furthermore, it serves as a basis for algorithmically determining the direction 
and magnitude of the sliding motion. 
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Assume e is the thickness of the hysteresis band around the sliding surface 
(see Fig. 17).^ If e is small, the helds f„ and ffs on either side of the surface can 
be represented by their point vector representations with normal components 
and fp, and tangential components and /^. The direction of movement is 
along the sliding surface, and we need to calculate the average velocity on the 
surface. 




Fig. 18. Filippov construction of sliding motion direction and magni- 
tude. 



The time taken by the system to cross the e band is 5ta = ^ in mode a, 
and dtff = in mode (3. The tangential distance the system travels over two 
adjacent time intervals {5ta + St/ 3 ) is: 

6 x = f^Sta + fffSt/j. (5) 

We compute the average velocity of the motion on the surface as: 



6tc 

= (f + f )/(* + *) («) 

= r/‘ +(l®r)/‘, 

where r = Thus, the vector v is on the line connecting the end points 

of fa and f/ 3 , with r and 1 r as its barycentric coordinates (Fig. 18). Let c 
be the difference vector f/ 3 ‘S>fa, and p be the intersection of c with the tangent 

vector. Let p partition c into two segments, d and e. We have | ^ by triangle 

^ This c has no direct relation to the coefheient of restitution used previously. 
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congruence. Thus, we have shown that the barycentric coordinate for p is the 
same as that for v (recall = j|-). This corresponds to Filippov’s construction, 
i.e., the vector v is the same as the tangent vector. 

In fact, the formula v = rfa + (1 ® can be used to compute v, where r 
is determined from the normal components of the two vector fields if the angle 
between the sliding surface and vector fields is known. 



Equivalence in Control Another approach to deriving the dynamics on a 
sliding surface is the method of equivalent control. For systems linear in control, 
this is an identical approximation to the equivalent dynamics [13]. However, 
nonlinear control may derive different behaviors since the true system behavior 
near the sliding surface can be attributed to hysteresis phenomena. Utkin [13] 
shows that the method of equivalent dynamics derives sliding behavior closer 
to the true dynamics than the method of equivalent control in these situations. 
In some cases, where system behavior is not well-behaved near the switching 
surface, e.g., the case of a saturated high gain amplifier, where system variable 
values tend to infinity close to the discontinuity, equivalent control may generate 
better approximations [6] . The reason is that there are no higher order hysteresis 
effects, therefore, modeling with equivalent dynamics, which assumes hysteresis, 
results in the generation of deviant behaviors. We also note that equivalent 
control methods can derive ehatter-free, equivalent control signals directly. In 
fact, the method of [17] exploits this property to obtain chatter-free simulation 
and control algorithms. 

It is critical to analyze the semantics of the discontinuity, and not introduce 
hysteresis effects in the model if they are not a true governing phenomena. 
Incorrect analysis may lead to models that are not well-behaved, and, therefore, 
cannot be correctly analyzed by equivalent dynamics. This issue needs to be 
investigated in future research. 



4 Sliding Mode Simulation Algorithm 

The sliding mode algorithm has been implemented as part of the hybrid system 
simulation engine [9] using a time-step halving forward Euler numerical inte- 
gration scheme. When a mode switch between fields /3 and a is detected (see 
Fig. 19), a binary search is invoked to determine the switching point to a pre- 
specified accuracy. After the mode switch is executed and the state vector is 
transferred to mode Xa(tg), it is checked whether the new mode, a, persists for 
at least one time step, AT. If it does not, the sliding mode simulation algorithm 
(Algorithm 4) is activated. The first step computes Xa(ts + AT) based on fa 
with Xafts) as the initial point. Discontinuous changes may take place, and the 
resulting xp{ts -h AT) is computed. xp(tf) is computed from Xa{tg) and fa- A- 
gain, xp{tf ) may differ from Xa{tf ) due to jumps in the state vector between a 
and (3. Next, xpflf -\- AT) is computed from xpftf ) and fp, and jumps between 
/3 and a arc taken into account when Xa{tf + AT) is computed. xp{tg + AT) 
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is taken as the initial point of the vector + AT) <S> xp{ts + AT). A binary 

search is performed along this vector given a pre-specified number of steps to 
determine x{ts + AT) on the sliding surface. After x{tg + AT) and the corre- 
sponding field are determined, checks are made to make sure chattering does not 
persist, and behavior continues to evolve in time. 



Algorithm 1. Sliding Mode Simulation 

Require; A mode switch has occurred from to /q. 

Compute Xo,{ts + ^T) from Xo,{ts) and fa(ts) 

Infer and Xf3{ts + A.T) {A discontinuous change between Xa{ts + ^' 1 ') and X0{ts + A. 1 ') may 
occur. } 

Compute Xoc{ts + e) from Xcc{ts) and fa{ts) 

Infer and X0{ts 4 - e) from Xo_{ts + e) 

Compute X0{ts + e + AT) from X0{ts + e) and f^its + e) 

Infer f^, and Xa,{ts + € + AT) 
while /cv 7^ f0 do 

Compute Xd — Xa.{ts + e + AT) — X0{ts + AT) 

9 — "2 1 Qaccuracy — ^ 
for a given number of iterations do 
Xe = Xffits + AT) + g * Xd 
Infer f~y from Xs 

J ~ fcx then 9 ~ 9 T 9a-ccurucy dse 9 ~ 9 9(iccuracy 
9 accuracy — 2 9 accuracy 

end for 

ts = ts + AT 
^a(ts) = 

if fc ^ A then = fp 

Compute Xct{ts + AT) from Xoc{ts) and foc{ts) 

Infer and x^its + AT) 

Compute Xa(ts + e) from Xoc{ts) and fo,{ts) 

Infer and X0(tg + e) from Xa(ts + e) 

Compute X0{ts + e + AT) from X0{ts + e) and f^its + e) 

Infer fc, and Xodts + c + AT) 

end while 



4.1 The Evaporator 

Consider the chattering behavior in Fig. 10 for the evaporator shown in Fig. 1. 
Simulation with time-step halving and sliding mode algorithm produces the re- 
sults shown in Fig. 20. The temporal, dynamic behavior of the system is no 
different but the error in the sliding mode is greatly reduced. This is further 
clarified in the phase space plots in Fig. 21. A fixed-step simulation produces a 
large error (same order of magnitude as the chattering) at p = 0 in determin- 
ing when the overflow mechanism becomes active, i.e., when L > 0.8. However, 
when time-step halving is used to accurately determine the switching surface, 
it reduces the integration step to the lower bound when chattering occurs. The 
system behavior is then determined using the sliding mode algorithm. 

4.2 The Cam- Follower System 

The results of numerical simulation of the cam-follower system is shown in 
Fig. 16. On the plot on the right in Fig. 16, the negative plane of the Crod ® Mam 
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Fig. 19. The sliding mode numerical simulation algorithm. 




Fig. 20. Simulation of the evaporator using the sliding mode algorithm. 




Fig. 21. Fixed step Euler (left) vs. time-step halving Euler with sliding 
mode simulation (right). 
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axis results in a non-elastic collision with Vrod — Vcam- This mode a, where the 
rod and cam system are connected, is active on the Vrod — Vcam axis as long 
as Fnormal is positive or 0. When F^ormal becomes negative, the system switch- 
es to mode /?, where the rod is disconnected from the cam and moving freely. 
For certain parameter values, this moves the system in the plane where a non- 
elastic collision occurs, and the system moves back to the Vrod ~ Veam axis, with 
Fnormal > 0. This chattering behavior is an artifact of the numerical time step. 
For AT 0, the system would remain at (0,0), corresponding to movement 
along the sliding surface, Vrod = Veam- Fig- 22 illustrates the operation of the s- 
liding mode algorithm. It shows that in mode (3 the rod and cam are disconnected 
and Vrod (thick lines) differs from Veam (thin line) . The non-elastic collision that 
follows immediately results in a jump from xp{ts -f e + AT) to Xa{ts + e + AT) 
which corresponds to Vrod = 'i^cam- Fi mode a, the rod and cam arc connected, 
and Vrod — Xcarn also. Therefore, the system slides on the switching surface and 
there is no error due to chattering. This agrees with the simulation results when 
higher order physical phenomena, such as adhesive forces between the rod and 
cam, are incorporated into the system model. 




Fig. 22. Sliding mode simulation for the cam-follower system. 



Simulation results using a fixed step Euler function produce the chatter error 
shown on the left plot in Fig. 23. The chatter error can be reduced by using a 
smaller step size, but time-step halving would increase the computational com- 
plexity significantly because the step size would be reduced to its lower bound 
very quickly during simulation. Sliding mode simulation accounts for the discon- 
tinuous jump in rod velocity. As shown on the right in Fig. 23, this produces 
an error-free behavior. Simulation results also show that the sliding mode simu- 
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lation correctly turns off when system behavior moves away from the switching 
surface. 




Fig. 23. Sliding mode simulation during an interval of time. 



5 Conclusions 

Modeling and analysis of hybrid systems present distinct challenges to researcher- 
s. Of particular interest are the development of numerical methods that generate 
accurate approximations of system behavior, especially at and around mode tran- 
sition boundaries where discontinuous changes can occur in system variables. To 
accommodate the steep slopes that can arise at the boundaries, simulation algo- 
rithms have to reduce step size to estimate slopes more accurately and generate 
more accurate behaviors. This paper extends our original hybrid system simu- 
lation algorithm to aceommodate an interesting class of behaviors, the sliding 
mode, that can occur at mode change boundaries. 

Sliding mode systems move along sliding surfaces because of continuous inter- 
action between two adjacent, alternating operating modes. However, modeling 
abstractions may lead to small, higher order dynamic effects such as parasitic 
inertia, capacitance, and resistance not being included in the systems model. 
Both the modeling abstractions and the discrete-time simulation can introduce 
chattering along the surface where system behavior seems to switch between t- 
wo modes of behavior at a very fast rate. As discussed, simulation errors can be 
kept small, by reducing the step size small in numerical integration so that fast 
chattering motions are not missed. The result is that simulated time progresses 
only in small increments, and the slower, sliding movement along the switching 
surface is not simulated efficiently. 

Based on a physical model semantics, we have developed a sliding mode 
simulation algorithm using the Filippov equivalence criteria applied to dynamic 
behavior generation. Our implementation has shown that the simulation main- 
tains consistency of the temporal behavior along the sliding surface while keeping 
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the error term small. The algorithm has performed well in several engineering 
applications where discontinuities in physical behavior result in mode switches. 
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Abstract. Hybrid transition systems in their full generality describe 
continuous behaviour by a set of equations in each mode - an algebraic 
or differential equation for each state variable in terms of inputs and 
other state variables. Each discrete transition may be taken according to 
a (non-deterministic) time constraint. 

In this paper we restrict this model to time-deterministic discrete transi- 
tions. Thus, every transition is guarded by a condition g and has a fixed 
delay t. Different transitions may have different delays (including zero), 
but progress is enforced after the delay. Using this restriction and a com- 
position operator which uses union of mode sets we then prove certain 
compositionality properties. In particular, that the parallel composition 
of two subsystems produces a system whose semantics is defined in terms 
of semantics of its constituents provided that it has a run. The restriction 
is well-justified in a large class of control applications where the complex 
mode-changing software is realised as a synchronous program. 



1 Introduction 

Hybrid theories combine the theory for continuous dynamic systems with the 
theory for discrete dynamic systems. While the theoretical work is carried out, 
safety-critical systems are being built and analysed at great costs. Successful 
application of formal verification techniques to such systems needs to start from 
engineering models of hybrid systems and systematically transform them to 
analysable models in the theory. The work presented here draws from the expe- 
rience with a number of case-studies, where the idea has been to start from the 
engineering models of physical systems “as they are” . 

Hybrid Transition Systems (HTS) were proposed in 1993 as an attempt to 
capture both mode- switching physical systems and non-deterministically timed 
computer systems in a single formalism. The formalism was studied in the con- 
text of several realistic examples: a two car no-collision scenario [17], a 16th 
century siphon pump machine [23,22], and the landing gear system of an air- 
craft [18,19] among others. Some of these examples were verified by the appli- 
cation of deductive methods. Simpler models were augmented by addition of 

* This work was supported by the Esprit LTR project 22703 (SYRF) and the Swedish 
board for technical research (TFR). 
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invariances after which they conld be model-checked as linear hybrid automata 
(LHA) [2], 

In this paper we propose a restriction of the formalism and prove certain 
compositionality properties, while keeping the expressive style so that engineer- 
ing models can still be plugged in with no additional effort. The motivation 
for the restrictions are twofold. First, a number of changes in the definition of 
the operational semantics and parallel composition operator facilitates proofs of 
compositionality. Second, restriction to time-determinism particularly suits anal- 
ysis of systems controlled by programs from the so-called synchronous family of 
languages [10], as described below. 

Hybrid transition systems in their full generality describe behaviours of sys- 
tems as interleavings of continuous phases of activity (having positive durations) 
and discrete transitions (taking zero time). Each discrete transition, however, 
may be taken according to a (non-deterministic) time constraint. A transition 
is taken a least I time units and at most u time units after it has been enabled 
(0 ool,u oooo). 

The work on case-studies suggests the use of time-deterministic control pro- 
grams in many applications, and the need for deductive reasoning about cases 
where models of the physical environment is non-trivial (eliminating the pos- 
sibility of model checking) [9]. We therefore propose to restrict the model to 
time-deterministic transitions while keeping the expressiveness with regard to 
dynamic continuous systems. Each discrete transition is thus required to be tak- 
en within a fixed period of time from the time at which it was enabled. However, 
in a composed system, the fixed periods may differ - due to transitions belonging 
to different subsystems. 

Hence, we expect to keep the best of both worlds in the following sense. 
Hybrid models are often built up from several modules each representing either 
a control unit or a physical apparatus (mechanics, hydraulics, electronics, etc.). 
For physical systems, our experience shows that models derived using systematic 
modelling (e.g. by bond graphs) are naturally described by switching phases 
of continuous activity, immediately on satisfaction of certain conditions (i.e. 
I = u = 0). For control programs, many realisations have a fixed period for 
each control function {I — u — d for some period d). Analysing the logical 
and causal behaviour of such synchronous programs [10] on their own is well- 
established [6,11]. By deriving the hybrid model we can now extend the analysis 
to the closed loop system behaviours — including assessment of “the synchrony 
hypothesis” . 

Safety properties of such closed loop systems can sometimes be verified on 
discrete models, but that requires the derivation of a discrete model of the en- 
vironment [25]. As synchronous languages are becoming more widespread in 
industrial systems, and interfaces to verification tools are forthcoming (see the 
Lustre-PVS connection for example [4]), it is even interesting to formally anal- 
yse the timing behaviour of such systems in relation to their environments. This 
generally requires continuous analysis combined with deductive proofs. The re- 
striction to time-determinism enables the direct translation of a synchronous 
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program to a time-deterministic HTS without transition delays (length of com- 
putation step). Adding transition delays and the model of the environment we 
get a model for the whole system to reason about. 

A different application of the time-deterministic model can be the composi- 
tion of timed models of synchronous subsystems, each having their own compu- 
tational period; the resulting system being a timed description of asynchronous 
networks of synchronous processes. Analysis of such applications is a subject for 
future work. 

The rest of this paper describes certain features of hybrid transition systems 
for natural modelling of engineering systems in section 2. Then follows the for- 
mal definition of time-deterministic HTS, and the parallel composition operator 
for this class of systems in section 3. Section 4 treats issues related to com- 
positionality and progress properties, and section 6 discusses relation to other 
works. 



2 Hybrid transition systems 

Hybrid transition systems treat the diserete and eontinuous elements in each s- 
tate on a par. That is, all elements of state are conceptually represented as piece- 
wise continuous functions of time. Each system has a finite number of modes. In 
each mode, the continuous elements of the state are concretely represented by 
a set of differential and algebraic equations in state space form (see 1), where a 
real- valued variable may change either in accordance with a differential equation 
or according to an algebraic equation in each mode. This facilitates modelling 
the cases where the dimension of the system is changed from one mode to anoth- 
er (see the siphon pump in [23]). In any case, the representation of continuous 
change as above provides a natural interface to engineering models which have 
this form, and which can be plugged into the hybrid model without additional 
transformation (e.g. addition of invariances and exit conditions as in LHA). 




Fig. 1. A schematic illustration of hybrid transition systems as a grap 



The discrete elements of state (piecewise constant functions of time) are 
represented as variables taking constant values in each mode. Thus, all changes 
in state are captured by differential and algebraic equations. Changes of mode 
are conditional upon a guard becoming true, i.e. a boolean expression g over the 
state and input variables. The change in input variables is not constrained. 
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The uniform treatment of continuous and discrete elements of state provides a 
natural means of communication between subsystems. Instead of shared labels or 
events we treat every state change as immediately visible by all other subsystems. 
Thus communication is by shared variables, where each variable is only allowed 
to be changed by one subsystem (the one whieh has this variable among its state 
variables). Other subsystems may (or may not) have this variable as an input 
variable - a model which supports non-symmetric communication (broadcast as 
opposed to rendezvous). 

The model is specially useful for modular developments. For each component 
the change in state is defined in terms of current state and the current input. 
The input is, however, not modelled explicitly (unless it is the state of another 
subsystem). This allows the proper treatment of disturbances or inputs whose 
modelling is to be postponed (e.g. the driver behaviours in [17]). 

Consider a mode change conditional upon the guard g and having a timing 
constraint [l,u], as depicted above. Then a watchdog for the guard can result in 
setting a timer to zero whenever the value of g changes to true. From this point 
onwards the transition may be taken, but at the point when the timer hits u 
the transition must be taken. In what follows, we consider the systems where 
all the transitions must be taken at the point which is exactly t time units apart 
from the time when their guard becomes true. 

3 Time-deterministic HTS 

We now define time-deterministic hybrid transition systems (TD-HTS) formally. 

Definition 1. A time-deterministic hybrid transition system is a tuple 
{M,X,U,F,I,T), where 

M: is a non-empty set of mode sets {mi, . . . , m^}. For a simple system m-t are 

singleton, for a composed system each mi is a set of mode elements. 

X: is a set of typed state variables with disjoint subsets Xc and X^, X — XeUX^, 
where the domain of Xc variables is M. The set of states of the system, S, 
is the set of type consistent interpretations of the variables in X . 

U: is a set of typed input variables U = UcUUd where the domain of the variables 
in Uc is K and X HU = 0. 

F : M ooX ^ £ is a function associating an equation with each state variable 
in each mode, where e = F{m,x) has the following forms: 

— if X <E Xd then e = '~x — c~' for some value c of the right type 

— if X G Xc then 

ooF{m,x) ='^x ~ f{x,u)^ for some function f of appropriate type, 
ooor F{m,x) = ^x — g{x,u)~' for some function g of appropriate type 
where x and u denote vectors of variables over Xc and Uc- 

I: is an initial configuration consisting of (mo, s'o), where mo G M and sq G S. 
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T: is a set of mode transitions r = (m, m' , g, t) where m, m' S M , g is a boolean 
expression over terms of the form zffw, with z G XUU , ff G { = , oq oq <, >}, 
and w is a type consistent term over X and U , elements from their value 
domains, and uninterpreted function symbols, t G N corresponds to an exact 
delay on the transition r. 



□ 

We denote by I{x) the value of the variable x in the interpretation I. The 
guard g is defined to be true in an interpretation X if it evaluates to true when 
all the variables 2: in g are substituted by the values given by T{z). 

The operational semantics can be informally described as follows. A run for 
a system is defined from the initial configuration, for a given input time-function 
(a function from the reals to interpretations for the input variables). The run 
consists of a sequence of mode-state-input-time tuples where the changes in 
the input components are governed by the input time-function at selected time 
points. The recorded time points are those at which a mode change takes place, 
or some guard to some transition changes its truth value. The state changes 
are compatible with the solutions to equations in each mode. The (discrete) 
mode changes are recorded by having the same time component and different 
mode-state components (though the value of continuous state variables are un- 
changed at mode changes). Such a mode change appears in a run if the guard 
g for some transition r has been true for the duration t as dictated by r. Al- 
1 enabled transitions take place at the end of their respective durations t (i.c. 
they must take place). If there are several such transitions, then one will appear 
non-deterministically in the current position in the run, and the others will also 
take place (with the same time component but at the next position in the run) 
provided they are still enabled in the current mode. 

Next we give a formal account of the operational semantics. In definitions 
below we represent the interpretation of a set of variables Z as the set {{z,v) \ 
z E Z}, where each u is a value of the right type. 

Definition 2. Let {M, X,U, F, I ,T) be a time- deterministic hybrid transition 
system, and 7 denote a finitely variable function from K to the set of interpre- 
tations over U . Then the run of the system with input 7 is an infinite sequence 
of situations ctq, cti, . . . such that: 

— Gj — {ruj, Sj,ej,tj) where Cj = ditj) 

— to,ti, . . . IS a progressive time sequence with tj eM. 

— GO = (mo. So, 7(0), 0) 

— each state component Sj G S, and is compatible with the trajectory of the 
system, in m.ode mj - i.e. Sj{x) = 'Jxitj) where 7^, denotes the solution 
to the DAE in mode mj as defined by F with respect to variable x, given 
the initial values corresponding to Si and Ci from the last position i in the 
run when the mode changed, a position i < j, such that rrii -fi mj, and 
yk i < k 00 j mi- = rrij 
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— at every tj either the mode changes, or the guard to some transition changes 
truth value, i.e. rrij = Wj+i ^ 3 {m,m' , g,t) G T such that g is true (false) 
in Sj U 6 j and false (true) in s^+i U Cj+i 

— mode changes take no time, i.e. ruj ^ nij^i tj — tj^i 

— a tra,nsition is taken only if it is enabled for long enough, i.e. 
mj 7^ mj+i ^ 3 t = (m, m', g,t) G T 3 i ooj such that 

Vi oofc ooj, g is true in Sfc U Cfc, m D mfc 7^ 0 and m' D 7^ 0 

— for every transition with guard g and delay t, if g becomes true at a time point 
ti and stays true at all subsequent situations prior to the time point tj = tj+t, 
then the transition is taken at tj provided that it can be taken , i.e. Vr = 
{m,m' , g,t) , iftj — ti + t, g true in SiUCi, andVi 00k ooj, g true in SfeUcfc 
and m fl mfc 7^ 0 then 3 a — (m" , s,e,tj) where m' H m" 7^ 0 , s(x) — Sj(x) 
for all X e Xc, and is otherwise defined by F(m" ,x) (for x G Xj). 

□ 

Note that we want to force progress as a rule. So any transitions which 
have been enabled “for long enough” must be taken. On the other hand, we do 
not want to force determinism — specially since parallel composition of several 
subsystems may lead to independent transitions getting ready for being taken 
at the same time. The semantics thus allows for all such transitions to take 
place in an arbitrary order but at the same time point provided that they are 
not in causal conflict with each other. The latter is not explicit in the semantic 
definition but is implicit: a system which does not have a run according to the 
above definition does not have a well defined semantics. The detection of causal 
paradoxes , e.g. that a taken transition disables an already enabled transition, 
may be investigated using similar procedures to those for synchronous systems [6] 
- taking no regard to the timing delays. 

Next we give the definition for composition of TD-HTS. Note that the com- 
position operation is dehned provided that a condition on shared variables holds. 
This might be slightly problematic with certain systems in which common out- 
puts are desired, e.g. subsystems which both emit the same alarm signal in 
different situations. This situation can however be remedied by renaming of the 
state variable in one subsystem after composition. 

Definition 3 . Given time-deterministic HTS H\ — {Mi,Xi,Ui,Fi,li,Ti), and 
Fl2 = {M2, X2,U2, F2, l2,T2) , such that X\ n X2 = 0 and M\ n M2 = 0 , we 
define their composition denoted by H1WH2, as the time- deterministic HTS 
H = (M, X, U, F, /, T) where: 

M = {mi U m2 I mi G Ml, m2 G M2}, 

X =Xi UX2, 

U ={UiU U2) ® X, 

F : M ooX ^ E where E is the set of equations E\ G E2, such that 
F{m, x) = Fi{mi, x), for x G Xi, m^ com G M, m^ G Mi 
(* € { 1 , 2 }), 

T is the smallest set of transitions such that: 
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if (mi, m[,gi, ti) &T\, then for all m S M2, we have 
(mi U m, m[ Llm,gi, ti) G T, and 
if {‘m,2, m2, 52, ^2) C ?2, then for all m G M\, we have 
(m U m2, m U m2, < 72 , ^2) G T. 

□ 

Proposition 1 . The parallel composition operator is commutative and associa- 
tive. 

Proof. Follows directly from the definitions and the properties of set union. 

□ 



4 Compositionality of TD-HTS 

We are interested in compositionality of hybrid systems in two different ways. 

— Proving assertions of the type “if Hi and H2 satisfy a particular property, 
so does H1WH2'” . 

— Proving that “if H1WH2 satisfies a property P and Hi is equivalent to H^ in 
some sense, then Hs\\H2 satisfies P. 

The first one is needed for bottom-up modelling and verification of system- 
s, and the second for making simplifications prior to analysis, or for refining 
abstract designs to implementations. In this paper we treat the hrst aspect of 
compositionality. That is, we show that composition of systems preserves certain 
interesting properties. A central property for hybrid transition systems is having 
a well-defined semantics in terms of a set of runs. Thus, we would like to show 
that if Hi and H2 each have well-defined semantics in terms of a (non-empty) 
set of runs, then their composition also has a run. Note that the definition of 
a TD-HTS does not guarantee existence of a run for the system. In particular, 
systems with obscure behaviours (e.g. the so-called Zeno behaviours in which 
time progresses in small increments but never beyond a particular bound) do 
not have a run, but are not syntactically excluded. Unfortunately, we can not 
exactly characterize systems whose states are finitely variable (have finite num- 
ber of discrete state state changes in a finite interval of time), or compositions 
of systems which have no runs. 

Therefore, we proceed as follows. First, we distinguish systems in which an 
infinite number of discrete transitions are possible in a single point in time. The 
intuition for this rests on experience with modelling realistic systems. Models 
of physical systems which are derived with systematic modelling techniques do 
not exhibit such behaviours, despite the fact that in these systems, structural 
change can be naturally modelled by an instantaneous transition [ 23 ]. Models of 
computer systems are definitely non-Zeno as long as there are non-zero delays 
associated with every transition. Therefore, we define the notion of admissi- 
ble systems which at least do not exhibit the undesired behaviour of changing 
infinitely often in a single time point. 
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Definition 4. Let H — {M, X,U, F, I ,T) be a TD-HTS. We associate a la, helled 
graph Gh — E, L) with H such m € M iff Vm. S V , and (t>, v') G E iff 3 t = 
{m,m' , g,t) G T; the labelling function L associates the table (g,t) with the edge 
define the TD-HTS H as admissible iff for every cycle in Gh, 
if all labels are of the. form, {g,0), then there are at least two edges in the cycle 
with guards g and g' , where g and g' are mutually exclusive when evaluated in 
all interpretations of X UU. 

□ 

Proposition 2. If H\ and H2 are admissible then H1WH2 is admissible. □ 

Obviously, this property does not guarantee existence of a run when two sys- 
tems are composed, but it eliminates some cases which depend on inappropriate 
enabling of guards. 

Next we show that the parallel composition of Hi and H2, produces a system 
H whose semantics is defined in terms of semantics of its constituents provided 
that it has a run. That is, a sequence belongs to the set of runs of H if it is 
grounded in runs of H\ and H2 in a sense that we make more precise as follows. 

Proposition 3. Let H\ and H2 he two TD-HTSs. Let 71,72 and "f be three 
input time functions with the following properties: 

1. Hi has a run with 71, i G {1, 2}. 

2. ’y{t){u) = 7i(t)(u) if u gUi® X2, and 'f{t){u) = 72(t)(w) if u GU2® Xi. 

3. ji{tj){u) — 6j{u) at every position j of the run for H2 with ^(2 ifu G Uir]X2, 
and j2{tj){u) = ej{u) at every position j of the run for Hi with 71 if u G 
U2 nXi. 

If H = H1WH2 has a run with 7, then for every such run of H , there is some 
run of Hi with 71 (and H2 with 72 j such that for every element at position i in 
the run of H there is a corresponding element with the same time component ti in 
the run of Hi{orH2) such that the restriction of the element in H to (state, mode 
and input) variables of Hi{orH2) gives the element in the run for Hi{orH2). 

□ 

Proof. By induction on the sequence of situations in the runs of H. Consider an 
arbitrary run of H with 7. 

Base step: for the initial state the condition is trivially true. 

Induction step: Consider a fragment of the run: . . . ,ai, CTj+i, . . . 

We will show that for any element a^^i at position z + 1 there is an element 
related to it at a position j < z + 1 in the run for H , such that, if there is an 
element corresponding to Uj in some run of Hi with 71 (or some run of H2 with 
72), then there is an element corresponding to Uj+i in that run of Hi (or H2). 
There are two cases: 



1. mi rrii+i 

According to the operational semantics for H there exists a transition t = 
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{m,m' , g,t) in T such that rrii H m ^ 0,mt-|-i = m' , and g has been true 
in every Sfc U ek since a position j in which g became true. Assume that Uj 
restricted to variables of H\ (or H 2 ) exists in a run for H\ (or i? 2 )- Then 
according to the operational semantics, a situation corresponding to Ui+i 
with the same time component exists in the run of H\ (or H 2 ). 

2 . irii — m,+i 

According to the operational semantics there is a transition with guard g in 
H such that g is false (true) in s^Uei and true (false) in Si+iUej+i. According 
to the definition of parallel composition this transition (guard) can be traced 
to one of the two systems Hi and H 2 ■ Without loss of generality assume that 
it comes from Hi. Consider a run for Hi with 71 . Assume that there is a 
situation ak with time component equal to ti in this run for Hi, whose mode 
and state components are restrictions of rui and Sj to variables of Hi. We 
have to show that Cfc+i in that run corresponds to a restriction of Uj+i to 
variables of Hi. First, we show that g is false (true) in Sk U Cfc. This easily 
follows since is a restriction of Si to Xi, variables in g are a subset of 
Xi U Ui, and due to conditions 2 and 3 in the antecedents of the proposition, 
the value of these variables is the same as that in U Cj. 

Next we show that g changes truth value at Cfc+i and that takes place at 
time point ti+i. Again, since the variables in g are a subset of Xi UUi, based 
on definition of F in parallel composition, and restrictions 2 and 3 in the 
antecedent to the proposition, we can state the following: if is the first 
time point after in which g changed truth value in the run of H, then 
is also the first time point after Ufc in the run of Hi in which g changes truth 
value. Hence, according to the operational semantics, there exists a situation 
CTfc+i = {m' , s' , e' , ti-f-i) in the run of Hi such that m' 00 m^, s' = 
restricted to Xi, and e' — 7 i(t,;+i). 



□ 



5 Current application 

In the SYRF project we are investigating formal verification of a climatic cham- 
ber case study provided by Saab Aerospace. The physical environment consists 
of a chamber with two outlets, a fan to change the flow of air in the chamber, 
and a heater to warm the incoming air. The objective of the controller is to keep 
the temperature and flow of air in the chamber within predefined distance from 
dynamically changing reference values. To ensure that these objectives can be 
met the system performs continuous regulation as well as monitoring, the latter 
giving rise to several modes of operation. 

The controller is realised as a synchronous program which can be represented 
in a statechart-like model with different regulation activities associated with 
different modes. The controller program (or its functional specification) is too 
large to be included in this paper. The physical environment has so far been 
modelled with certain simplifying assumptions - nevertheless giving rise to the 
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following non-linear equation describing the change of chamber temperature in 
terms of the flow and the input heat power. Here Tcham,b denotes the inside 
temperature, Tj„ the incoming air temperature, u^eat the voltage applied to 
the heater, and q the (volume-based) flow of the air in the chamber. The k- 
terms are constants determined by the range of values involved and the chamber 
characteristics. 



Tchamh — j kchamb 

( Q Tin ^in T 

‘^heat ^heat ® 

TchambiSl ^out ^ klo 



s)) 



Further details of this case study which gives rise to a non-trivial hybrid 
system can be found in [20] . The physical environment model is modular in the 
sense that it excludes elements such as events (thresholds) used by the controller 
and dictated by the requirements specifications. 

The initial modelling step using composition of a synchronous program (with 
fixed delays) and the physical model is facilitated by the TD-HTS model. The 
next step is transformation of this model to a form which is directly analysable 
with existing formal verification tools. These are the type of models usually 
assumed as given in the verification literature. Analysis of the properties of the 
system and formulation of intermediate invariances is currently in progress. 



6 Related works 

We share a common aspiration with data flow approaches like Signal [5] : that of 
treating the discrete and continuous elements on a par. 

Hybrid transition systems can be seen as a modular version of phase transi- 
tion systems [15], our main contribution being the treatment of compositionality 
and separation of input and state components. Other versions of phase transition 
systems [16] include the notion of important events which, as well as enabling 
conditions (corresponding to our guards), include assertions from the require- 
ments specification for a system. Hybrid transition systems do not mix the model 
of a system and its requirements. 

Hybrid automata [2] differ mainly on the model for communication and par- 
allel composition. There, communication is by shared synchronisation labels. 
Moreover, composition is defined on systems which have the same set of con- 
tinuous variables, making modular models of different subsystems less natural. 
Hybrid automata also require invariants in order to force progress. This is often 
a significant additional modelling step not directly present in engineering mod- 
els. In our model, progress is by default, and addition of invariances are seen as 
part of the verification process. 

Modular modelling appears in several other frameworks, e.g. in [24,1]. The 
work by Westhead and Hallam [24] models hybrid computations as a limit to 
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discrete synchronous computational processes. Abadi and Lamport [1] discuss 
composition of, and decomposition into, specification modules. Their specifi- 
cations are representations of safety and liveness properties in temporal logic. 
Separation of input and state arises naturally in models proposed within control 
theory, e.g. several models discussed in [7]. However, rather than composition- 
ality, aspects such as stability are in the focus of these discussions. 

A hybrid theory which treats aspects of compositionality can be found in 
the work by Lynch et.al. [14] wherein both communication by shared variables 
and shared labels is considered. This work also attacks proving non-Zenoness by 
switching to a game theoretic framework. 

Going to the domain of timed systems, there are works which treat compo- 
sition in presence of non-deterministic timing constraints in variants of timed 
automata [21,12]. Sifakis and Yovine distinguish between transitions which must 
take place and those which may take place by using a notion of deadline in ad- 
dition to the usual notion of invariance in timed automata. Kesten, Manna and 
Pnueli, on the other hand, separate the progress conditions and enabling condi- 
tions by using different assertions for each. The former being global assertions 
and the latter as local predicates on each transition. 

7 Summary and future works 

The paper presents a model for hybrid systems where all elements of state (dis- 
crete and continuous) are represented as state variables. Modes of continuous 
activity are governed by differential and algebraic equations in state space form, 
and communication is by shared variables. The model is a modification of hy- 
brid transition systems presented earlier — the modifications being restriction 
to time-determinism and composition by union of mode sets. The restrictions 
are based on practical experience with modelling realisic engineering systems, 
and an intention to link to verification efforts on the synchronous family of lan- 
guages. A treatment of compositionality for open subsystems was discussed and 
a proof of preservation of semantics on composing subsystems was presented. 

Future works include continued investigation of the above industrial case s- 
tudy, and the investigation of how proof techniques from computer science and 
control theory can be combined in systems having diverse requirements specifi- 
cations. 
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Abstract. Given a heuristic estimate of the relative safety of a hybrid 
dynamical system trajectory, we transform the initial safety problem 
for dynamical systems into a global optimization problem. We compare 
untuned performance of several Simulated Annealing and Multi Level S- 
inglc Linkage method variants, and discuss the dynamic use of knowledge 
gained during optimization. 



1 Introduction 

Given a simulated hybrid dynamical system S, a set of possible initial states I, 
and a set of “unsafe” states U, we wish to verify nonexistence of an 5'-trajectory 
from I to U within tmax time units. We call this the initial safety problem. Sup- 
pose we are given an approximate measure of the relative safety of a trajectory. 
More specifically, let / be a function taking an initial state i as input, and eval- 
uating the S trajectory from i such that /(i) = 0 if and only if the S'-trajectory 
from i enters U within tmax time units, and f{i) > 0 otherwise. Then verifica- 
tion of the initial safety problem can be transformed into the global optimization 
(GO) problem: 

min(/(j)) > 0 
* 6 / 

GO methods may therefore terminate when i is found such that /(*) = 0. 
Given that / does not generally have an analytic form, we do not assume the 
availability of derivatives. Since each evaluation of / may require a computation- 
ally expensive simulation, we are particularly interested in GO methods which 
perform relatively few evaluations of /. In this context, we compare several 
variants of Simulated Annealing (SA) and Multi Level Single Linkage (MLSL) 
methods and assess their suitability for our purposes. We discuss the use of 
knowledge of / gained in the course of GO, and consider the extent to which 
some GO methods assume special properties of the local optimization (LO) pro- 
cedures they use. 

* This work was supported by the Defense Advanced Research Projects Agency and 
the National Institute of Standards and Technology under Cooperative Agreement 
70NANB6H0075, “Model-Based Support of Distributed Collaborative Design”. 
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2 Motivation 

Our research was largely motivated by the following safety veriheation task: Giv- 
en bounds on the system parameters of a stepper motor (e.g. viscous friction, 
inertial load), bounds on initial conditions (e.g. angular displacement and veloc- 
ity), and an open- loop motor acceleration control, verify that no scenario exists 
in which the motor stalls. We model the motor’s continuous dynamics using 
ODEs given in [1]: 

0 — LO 

(<8>ia-^b sin{N9) + ihN^ cos{N0) ® D sin(4fV0) (g) F^oj ® Fcsign(w) ® Fg) 

(Jl -|- Jm) 

*a = (Va 0 ia.R + ujNh sm{N 6)) / L 

*b = (kb 0 ihR 0 cos{N6)) / L 

where 9 and uj are motor shaft angular displacement and velocity, ia and ih are 
coil A and B current, 14 and 14 are coil A and B voltage, R and L are coil 
resistance and inductance, N is the number of rotor teeth, is the maximum 
motor torque per amp, D is the maximum detent torque, Fv is the viscous 
friction, Fc is the Coulomb friction, Fg is the gravitational torque load, and J\ 
and Jm are load and motor shaft inertia. For this system we classify a stall as 
deviation of or more radians from the current desired 9 equilibrium. 

The motor is stepped by reversing polarity of the coil voltages in alternation 
(see Figure 1). 




Fig. 1. Simple Stepper Motor Stepping 



Changes to coil voltages occur on such a small time scale that their contin- 
uous simulation is judged unnecessary for modeling dynamics relevant to the 
veriheation task. Voltage changes are therefore approximated as discrete events. 
Our acceleration control is open-loop: At hxed intervals the motor is stepped 
according to an acceleration table. We can express such a system as a nonlinear 
hybrid automaton as shown in Figure 2. 

First, we note that there is no apparent “geometrically linear hybrid system” ^ 
approximation with which we could apply the tools of computational geometry, 

^ i.e. restricted to constant first derivatives; “geometrically” as opposed to “alge- 
braically” 
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T = 0 
Va := 
Vb := 




• • • 



Fig. 2. Stepper Motor Nonlinear Hybrid Automaton 



but simulation is feasible. Next, we note that our verification is concerned with 
a fixed initial time interval (i.e. during acceleration) and is therefore an initial 
safety problem. Finally, we note that we can compute minimum angular dis- 
placement from a stall state over all simulation states as a simple heuristic to 
numerically rate the relative safety of safe trajectories. We can now ask, “For 
all possible system parameters and initial states, are all simulation trajectories 
rated safe?” Put another way, “Is the minimum heuristic evaluation of all possi- 
ble simulations greater than zero?” If we can answer this optimization question 
positively, we have verified safety of our hybrid system. 

One could argue that such optimization is not verification, that one cannot 
exhaustively simulate all possibilities and can therefore have no guarantees. One 
can only use such optimization for refutation. To this, we offer two responses; 
First, if one has additional knowledge of characteristics of one’s henristic eval- 
uation function (e.g. Lipschitz conditions), then an intelligent optimization ap- 
proach can utilize such characteristics to guarantee a strictly positive minimum 
with enough testing. The key is to provide a heuristic evaluation that induces a 
helpful search landscape without itself become overly burdensome computation- 
ally. Second, if one has no such knowledge about the heuristic, the absence of 
verification techniques well-suited to non-trivial dynamics leaves good global op- 
timization as our best assurance. Our desire is to develop an information-based 
GO method which, when halted without finding an unsafe trajectory, provides 
some measure of the thoroughness of its search. 

This said, we have endeavored to study a number of representative global 
optimization techniques in order to assess their suitability to our purpose and 
point the way towards future innovation. 

3 Algorithms and test problems 

In this section, we describe the global optimization (GO) algorithms used in this 
study, the local optimization procedures used by them, and the test functions 
to be minimized. Author-supplied default settings were used for GO algorithms 
when possible. Otherwise, reasonable parameters were held constant throughout 
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testing. Since onr goal is to perform a computationally expensive optimization, 
we would desire an algorithm which reliably and efficiently gives the desired 
result without tuning. Experienced users of such algorithms applying problem- 
and domain-specific knowledge to the choice of options and parameters could 
expect to yield better results. 

The first set of afgorithms we consider are variants of simufated annealing 
(SA) [2,3]. SA algorithms are theoretically guaranteed to find the gfobaf mini- 
mum of a function provided that the annealing schedule starts with sufficiently 
high temperature and cools sufficiently slowly. However, this guarantee conies 
at great expense in terms of function evaluations. Finding a suitable annealing 
schedule which balances the tradeoff of reliability versus efficiency is key to the 
practicality of SA for our purposes. 

AMEBSA [4, pp. 451-455] performs SA by modifying a downhill simplex method 
[4, pp. 408-412] such that actual function values of simplex points and possible 
replacement points are perturbed according to the temperature parameter when 
making move decisions. Since AMEBSA has no default annealing schedule, we 
have chosen to use the one supplied in the authors’ example [5, pp. 182-184]. 
ASA^ [6] , “adaptive simulated annealing” , is a S A variant that relies on randomly 
importance-sampling the search space and adapts separate annealing schedules 
for each parameter. The automatic adaptation of the annealing schedule trades 
off reliability for efficiency. SALO [7] seeks to combine the theoretical guarantees 
of SA with the efficiency of local optimization (LO). SALO on / is SA on /', where 
f is / transformed by LO. At each point that SA evaluates, LO takes place and 
the value of the local minimum is returned. This is intended to “flatten” / and 
speed convergence to the global minimum. In both implementations described 
here and in [7], ASA is used as the SA method. In doing so, we again tradeoff 
reliability for efficiency. When each of these SA methods halts unsuccessfully, it 
is restarted from the lowest point found thus far. 

The second set of algorithms we consider are variants of Multi Level Single 
Linkage (MLSL) [8]. MLSL uniformly, iteratively samples the search space and 
performs LO selectively. For each iteration, a new batch of points is evaluat- 
ed. For each point sampled, LO takes place if there exists no lower sampled 
point within a critical distance.^. MLSLl is the original algorithm [8]. MLSLD is 
a variant of MLSLl which assumes that the LO procedure is deterministic and 
should therefore never be repeated from the same sampled point. MLSLO orders 
optimizations for each iteration by ascending function value of sampled points. 
MLSLOD has both variations. MLSLSA alternates iterations of MLSLOD with runs of 
ASA, using the current minimum as the initial point for ASA. LMLSL is a variant 
of MLSLl which performs “lazy” function evaluation. That is, the function value 
of a point is only evaluated when it becomes necessary. This avoids the rela- 
tively large initial cost when optimizing simple functions. LMLSL^ is LMLSL using 

^ ASA software developed by Lester Ingber and other contributors is available at URL 
http : / /www . ingber . com/ or f tp : //f tp . ingber . com. 

® We used the critical distance parameter ( = 2 with 100 points generated per itera- 
tion. 
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an e-descent LO procedure. An e-descent procedure guarantees that, for a step 
greater than e, the function values at e-intervals along the step are sequentially 
descending. 

RANDLO simply performs random local optimizations and is intended to pro- 
vide a baseline for understanding how well LO knowledge is used by SALO and 
MLSL methods. MDNTE is a Monte Carlo method. 

We next describe the local optimization procedures used by some of these 
global optimization algorithms. FMINU and CONSTR are Matlab™ optimization 
functions [9]. FMINU performs unconstrained optimization using a quasi-Newton 
method with a BFGS formula for updating the Hessian matrix approximation. 
FMINUg is an e-descent modification of FMINU. CONSTR performs constrained op- 
timization using a sequential quadratic programming method. We supply search 
space bounds and no additional constraints. YURETMIN is a variant of Yurct’s 
Masters thesis Procedure 4-1 [10, p.33] which allows specification of search s- 
pace bounds. 

Finally, we reference the objective functions used for comparing the global 
optimization algorithms. The first part of our study uses functions selected from 
GO literature and algorithm demonstrations in order to reveal their relative 
merits. RAST is a scaled Rastrigin function [7]. HUMP is the six-hump camel- 
back function [11]. G-P is the Goldstein-Price function [11]. GWl and GWIOO are 
6-dimensional Griewank functions with bounds of each dimension [01,1] and 
[0100, 100] respectively [7]. SWISS is a 4-D paraboloid with a lattice of many cir- 
cular pits [5]. CMMR is a 4-D paraboloid with a grid of deep troughs [12]. GWIOO, 
SWISS, and CMMR have many local minima. RAST has a moderate number. HUMP, 
G-P, and GWl have few. RAST, GWIOO, SWISS, and CMMR are generally paraboloid 
in shape with different local minima “traps” . All slope up to the bounds of the 
search space. 

The second part of our study concerns the motivating example for this re- 
search. Test function STEPl takes as input two parameters (viscous friction and 
load inertia) of the stepper motor modeF, simulates acceleration of the motor, 
and performs a simple heuristic evaluation of the trajectory by computing the 
minimum distance to a stall state (0 if stalled). Such a heuristic function is often 
simple to construct. STEP2 is STEPl logarithmically scaled so as to focus on the 
unsafe region of the parameter space. These functions are shown in Figures 3 
and 4. 



4 Results 

Our first tests made use of LO procedure FMINU where applicable. 100 opti- 
mization trials were performed for each objective function with a maximum of 
10000 function evaluations permitted per trial. Each objective function was off- 
set (if necessary) to have a global minimum value of 0. A successful trial was one 

All other parameters and initial conditions are fixed. Given that the original problem 
posed was too easy, the search space was restricted to provide greater challenge for 
the methods under comparison. 
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Fig. 3. Stepper Motor Test Function STEPl 
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in which the optimization routine found a point with function value less than 
.001 within 10000 function evaluations. This was intended to simulate conditions 
where the search space contains very small regions with unsafe trajectories. 

Each entry in the table of results (Figure 5) shows the number of successful 
trials (upper left) and the average number of function evaluations for such trials 
(lower right). 

Giving the best performance in nearly half of the tests, RANDLD performed 
surprisingly well, especially for SWISS which has a 4-D lattice of numerous “trap- 
s’’ . As RANDLO’s LO procedure, FMINU is clearly rarely caught in such traps. Since 
both trap and non-trap regions are paraboloid surfaces, they effectively “point” 
to the global minimum for LO procedures such as FMINU. The simple but im- 
portant observation here is that local optimization does not necessarily find 
the nearest local optimum. We next observe that both SALO and MLSL each 
rely somewhat on nearness of LO, and will later turn our attention to the re- 
lationship between the global and local layers of each. FMINU, which assumes 
/ is continuous, behaved understandably poorly for highly discontinuous CMMR. 
Thus all methods dependent entirely on LO failed all CMMR trials. Given that the 
characteristics of / may not be well understood, this means that a less efficient 
LO procedure making fewer assumptions would likely be better suited to our 
purposes. 

SALD yielded performance similar to that of RANDLO where few LOs sufficed 
and significantly better where more local optima trapped LO (e.g. in RAST and 
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Fig. 4. Stepper Motor Test Function STEP2 
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GWIOO). At the heart of SALO’s design is the following intention: “SA helps in 
locating good regions of the search space, while the local optimizer is used to 
rapidly hit the optimum.” [7] It is clear from this comparison that SALO does 
indeed successfully apply SA on /' to hnd good regions of /. When comparing 
ASA with SALO, it also appears that the cost for transforming / ^^5*^ /' is usually 
more than compensated by the efficiency gained. 

SALO was designed with hope that /' would be a “simpler” surface than /, 
reflecting the function value of the nearest optimum. Interestingly, the designers’ 
experiments utilized Yuret’s LO procedure which has short term memory and 
takes increasingly greater steps downhill as success allows. Such a LO procedure 
can possibly pass over nearest local minima as step size becomes large. Also 
Yuret’s procedure, being stochastic, doesn’t simply transform one surface to 
another. Nevertheless, their experiments and ours indicate that ASA is able to 
handle such LO output gracefully in the long run. The fact that SALD outperforms 
RANDLO for harder optimization problems is specihcally a property of SA and 
more generally a form of learning. One can view the changing state probability 
distribution of SA as a gradual accumulation of knowledge about the location of 
the global minimum. While such learning is effective given a suitable annealing 
schedule, it is also weak. Heavily traversed local minima may be heavily traversed 
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404 
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100 
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100 
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2 
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100 
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100 
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SALO 
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65 


100 

97 


100 

85 


95 
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100 
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0 

N/A 


MLSLl 
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100 
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100 
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47 
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100 
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0 

N/A 


MLSLD 
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100 
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60 

4492 


100 
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0 

N/A 


MLSLOD 
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100 

132 


100 
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52 

4370 


100 

253 


0 
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MLSLSA 
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100 
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100 

130 


100 
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22 
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100 
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99 

5019 


LMLSL 


100 
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100 
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100 

118 


100 

96 


50 

4508 


100 
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0 

N/A 


LMLSLj 


100 

638 


100 

96 


100 

109 


100 

93 


53 

3864 


100 

192 


0 

N/A 


RANDLO 


100 

706 


100 

70 


100 

96 


100 

85 


58 

4008 


100 

146 


0 

N/A 



Fig. 5. Successful global optimization trials and average function evaluations 

again. All but one of the function evaluations made in LO are ignored. Much 
information is wasted. Nonetheless, SALD’s performance was impressive. 

Performance of MLSL methods, though similar to that of RANDLO, yields 
little to commend them over RANDLO. That selective uniform random LO should 
perform worse than unselective uniform random LO suggests an assumption in 
MLSL which is not met in our study. Following the analysis more closely in both 
[13] and [8], we see that MLSL’s LO procedure is assumed to be an e-descent 
procedure such that the current critical distance effectively bounds the step size 
of LO.^ We therefore created FMINU^, an e-descent modification of FMINU, and 
tested LMLSLj for comparison. Although LMbSL^ is somewhat of an improvement 
over LMLSL, it is still generally worse than RANDLO. e-descent does not therefore 
appear to help us much. We conjecture that MLSL methods dominate RANDLO 
for objective funetions where LO is trapped in many minima, and that SALO 
dominates MLSL methods for such objective functions in our study because our 
/'-surfaces are easily globally optimized with LO. To elucidate the latter point, 
consider RAST, GWIOO, and SWISS. LO roughly transforms each into a paraboloid 
of plateaus. LO of such LO-transformed functions can then efficiently lead to the 
global optimum. We can view the task of global optimization as multi-level local 
optimization. The base-level LOq takes advantage of whatever information about 
/ is available (continuity, gradients, etc.), the next level LOi is suited to the class 

® This is nowhere mentioned in survey [14] and is not emphasized elsewhere in the 
literature. 
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of one’s LOo-transformed function /', and so on. We may stop after arbitrarily 
many (probably 2-3) LO levels and simply perform random local optimization. 
The role of each LO level is to enlarge the regions leading to global optima. We 
arc currently developing such multi-level local optimization methods. 

Regarding MLSL methods, let us also note that, like SALO, they all but ignore 
information gained through LO. Uniformly sampled points are locally optimized 
based only on the values of sampled points within a critical distance. Again we 
find great waste of information gained at great expense. 

AMEBSA gave mixed results which can likely be attributed to the lack of an- 
nealing schedule tuning. Perhaps an adaptive annealing schedule would make 
AMEBSA more suitable for such problems. ASA’s efficiency was unpredictable, al- 
though it was perhaps the most reliable method for this set of objective functions. 

While these functions may give a general indication of the relative strengths 
of these methods without tuning, the functions share a common property unde- 
sirable for our purposes: The unconstrained global minimum is never located at 
or beyond the bounds of the search space. Therefore, our optimization methods 
need not perform well along the bounds of our search space. It is for this reason 
that unconstrained FMINU was suitable for use with such global optimizations. 
We used this as an opportunity to try two constrained LO procedures CDNSTR 
and YURETMIN for the stepper motor test problems STEPl and STEP2. For this 
testing, we performed 10 trials to find a function value of 0 with a maximum of 
1000 function evaluations per trial. The results appear in the tables of figure 6. 
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(a) CONSTR (b) YURETMIN 

Fig. 6. Results for STEPl and STEP2 



Since both STEPl and STEP2 have a small number of local minima along the 
bounds of the search space, behavior of LO again figured most significantly in 
our results. Despite the fact that much of the search space slopes downward away 
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from the corner where failures occur, CQNSTR had a bias towards looking in that 
particular corner. It was thought that STEP2 (log-log scaled STEPl) would be an 
easier function to optimize, but this was not the case. Not only was the global 
minimum basin expanded, but nearby local minima also expanded, trapping LO 
more often. 

ASA’s function evaluation expenses were such that it was outperformed by 
MONTE. The remaining LO-based methods performed similarly overall. The cost 
of computing simple heuristie information about relative safety of trajectories is 
usually more than compensated by efficiency in discovering unsafe trajectories 
through optimization. For both LO procedures, RANDLQ gave best performance 
for STEPl, and LMLSL gave best performance for STEP2. Although there was no 
universal “winner” among global optimization procedures, it is encouraging to 
note that procedures such as SALQ and LMLSL could be run in parallel to achieve 
respectable, more reliable results. The choice of LO procedure proved very sig- 
nificant for performance, which again underscores the importance of developing 
robust, efficient LO procedures suited to large classes of functions. 



5 Conclusions 

A powerful approach to initial safety verification is to transform the problem 
into an optimization problem and leverage the power of efficient optimization 
methods. 

This is accomplished by 

— providing a good heuristic evaluation function /, 

— choosing an efficient local optimization procedure well suited to /, and 

— applying a global optimization procedure such as SALD with ASA or RANDLQ 

for which one’s local optimization procedure is well suited. 

For our stepper motor problem, we are fortunate to have system trajectories 
which arc continuous in the initial condition. Our heuristic function preserves 
such continuity, so we can use such knowledge when choosing our local optimiza- 
tion procedure. However, if one cannot assume a continuous heuristic function 
(as will often be the case with hybrid systems), one can use the same approach 
choosing global and local optimization methods according to one’s assumptions 
or lack thereof. 

While no global optimization procedure was generally dominant in our com- 
parative study, random local optimization seemed best suited for objective func- 
tions with few minima, and SALO with ASA seemed best suited for objective 
functions with many minima. By making use of ASA for SA, one both avoids the 
need to specify an annealing schedule and benefits from its relative efficiency 
among SA algorithms. Although one is encouraged to make use of ASA’s options 
to improve performance, we have not done so and have been pleased with most 
results nevertheless. 
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SALD and MLSL methods perform global optimization with global and local 
search phases, and rely on local optimization for efficiency. However, both meth- 
ods make little or no use of information gained in the course of local optimiza- 
tion. We believe that great progress will be made in global optimization when 
global optimization and local optimization are seamlessly integrated to share 
knowledge gained of /. Where evaluation of / is computationally expensive, it is 
worth computational expense to utilize such knowledge for the efficiency of global 
optimization. To this end, we are currently developing a set of information-based 
optimization techniques where each optimization step is chosen with respect to 
the information gained thus far. 
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Abstract. Hybrid systems made up of a continuous plant supervised 
by a discrete-event controller are considered in this paper. A set of op- 
timal feedback control functions is defined at the continuous level. The 
aim of the discrete-event controller, modelled by means of a finite-state 
automaton, is that of choosing, in the above mentioned set, the best con- 
trol action to be applied to the plant in dependence of the current plant 
conditions and of possibly occurred external events (hence, the overall 
control scheme can be viewed as a two- level hybrid control scheme). An 
invariance property of the hybrid control scheme is proved and extensive 
simulation results are reported, showing the effectiveness of the proposed 
methodology. 



1 Introduction 



The contemporary presence of several kinds of dynamic behaviours within the 
same dynamical system represents the major feature of the so-called hybrid 
dynamical systems. An hybrid system is typically made up of a continuous plant 
whose behaviour is logically controlled by a discrete-event supervisor. Many 
real systems are intrinsically structured as hybrid systems; examples include 
manufaeturing systems, computer communication networks, constrained robotic 
systems, process control systems, and transportation systems. 

In the last years, the attention of many researchers has, thus, been focused 
on the possibility of modelling, analysing, and controlling the behaviour of hy- 
brid systems. Modelling frameworks of hybrid systems have been proposed, for 
instance, in [1], [2], and [3], whereas in [4], the authors suggest a general hy- 
brid model also providing a survey of related results. As regards hybrid control 
schemes, many different approaches can be found in the literature. In [5] and [6], 
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some general concepts of the supervisory control theory suggested by Ramadge- 
Wonham are applied to hybrid systems. A sliding-mode control approach specif- 
ically developed for hybrid systems is proposed in [7], whereas a multilayer hier- 
archical control problem is applied in [8] to Intelligent Vehiclc/Highway Systems, 
a particular example of hybrid systems in the field of transportation. A further 
hybrid control scheme is, then, formalized in [9] by means of H°° control con- 
cepts. 

A special class of hybrid systems is represented by switched systems, defined 
as combinations of finitely many continuous dynamical systems. A first qualita- 
tive analysis of the behaviour of switched systems is addressed in [10] in which 
’’multiple Lyapunov functions” are used. An extensive study of stability criteria 
for switched systems is reported in [11] where the authors establish new re- 
sults regarding the stability and asymptotic stability of the equilibrium of such 
systems. These results are mainly relevant to the construction of Lyapunov func- 
tions which, as already done by the same authors for discontinuous dynamical 
systems [12], allow the establishment of sufficient conditions for stability and 
asymptotic stability of the equilibrium of the considered systems. 

The model adopted in this work is based on the modelling eoncepts pro- 
posed in [13] and [14], whereas the hybrid control scheme can be viewed as the 
juxtaposition of two control levels: a continuous-state feedback controller and a 
discrete— event supervisor. At the continuous-state level a set of optimal receding- 
horizon feedback control functions are defined; the discrete-event supervisor acts 
by choosing the best continuous— state controller depending on current informa- 
tions about the plant and about external conditions. The invariance properties 
of the control scheme formalized in this paper are investigated, and a simple 
analytical result is proved. 

The remainder of the paper is organized as follows. In Section 2, the hybrid 
control scheme is preliminarly described. A detailed description of the feedback 
control scheme in the continuous-state level is reported in Section 3, whereas the 
discrete-event supervisor and the interface between the two control levels are 
outlined in the subsequent sections. In Section 6, the above invariance property 
is stated and proved, whereas Section 7 presents extensive simulation results on 
a significant nonlinear control problem, showing the effectiveness of the proposed 
approach. 



2 Definition of the hybrid control scheme 

Let us consider the discrete-time dynamic system (in general, nonlinear) 

^t+i = , t = 0,l,... (1) 

where Xt E X oo K” and Ut ^ U oo K"* are the state and control vectors, 
respectively. We assume that the constraint regions X and U belong to the 
class Z of compact sets containing the origin as an internal point. Moreover, we 
assume that f [M" oaK™, M"] , with /(O, O) = 0 ■ 
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Fig. 1. Structure of the hybrid control scheme. 



The structure of the hybrid— control scheme is shown in Fig. 1. This scheme 
has some similarities and analogies with the one reported in [15]. 

As shown in Fig. 1 , three different levels are present: a continuous-state level 
(CSL), and interface level (IL) and a discrete-event level (DES). In the following 
three sections, a precise definition of these levels will be given. However, let us 
first describe the rationale for choosing such an hybrid scheme. In this respect, 
we would like to emphasize that the approach pursued in the present work is very 
prcliminar and, as already mentioned in the Introduetion, many more general 
schemes of hybrid controllers can be found in the literature. 

In very rough terms (as just mentioned above precise statements will be giv- 
en in Sections 3, 4 and 5), the hybrid— control scheme behaves as follows: a set 
r of feedback control functions 7 i(c^is given; each instance of the closed-loop 
system [/, 7 ^] in the CSL is characterized by some typical behavior Bi . In the 
IL, at each time instant t, some decision algorithm T> elaborates batches of 

state vectors , x^} and control vectors . . . ,Ut} thus generat- 

ing some aggregate information vector Xj . The discrete-event system A in the 
DEL reacts to the input vector Xt and to possibly occurred external events by 
maintaining or changing the control function 7 j . 

3 The feedback control scheme in the CSL 

In the present section, a precise characterization of the nonlinear feedback control 
scheme is given. To this end, let us define the class of control functions X as 

X= { 7 *: A, ^ e /} (2) 

where ocX, Xi X, ^ i & I and where / is a set of integers with card{I) = if 

allowing the choice of a single ji € X . Clearly, this definition of X is too abstract. 
Then, a more specific set of control functions is addressed. We consider the class 
of discrete-time receding horizon (RH) regulators (see, for instance, [16], [17]). 








Control Scheme for Nonlinear Discrete— Time Systems 



265 



To this end, for any i £ I , we need to define the following fimte-horizon cost 
function 



t+N'-l 

J PH 1 ^ \ P ^ ~ ^ ^ 5 ^ ^ ll^t+A'^ llp^ : t C 0 (3) 

J=t 

where u^r = col (uj, . . . , u^-) , h e [M”CK"*,M+], with /i(0,0) = 0., 

\\x\^pi = x'^P^x , a® G K is a positive scalar, P® e M®®^®® is a positive-dehnite 
symmetric matrix, and A^® is a positive integer denoting the length of the control 
horizon. Then we can state the following 

Problem 1. For any i E I and at every time instant t C 0 , find the RH optimal 
control law ^ U, where is the first vector of the control 

jp ttO jp jjO . prjO rp TfO , ... , /o^ 

sequence, T that minimizes the cost (3J 

for the state x^ E X . 

Remark 1. As will be clear in the following, for any given i E I , the final cost 
(FWxt+NWpi plays a key role in the development of stability results concerning 
the RH regulator, and is essentially a penalty function related to the relaxation 
of the terminal state constraint Xippi — 0, imposed in [18] and in [16]. In the 
following, a basic result will be reported, showing the effects of the choice of N' , 
a®, and P® on the stability and performances of the i-th RH regulator. As will 
be seen later on, this result will be crucial in order to define in an effective way 
the interaction of the three levels in the hybrid scheme. 

Remark 2. The statement of Problem 2 does not impose any particular way of 
computing the control vector as a function of xy . Actually, we have two 

possibilities: i) on-line computation. For any i E I , Problem 1 is an open-loop 
optimal control problem and may be regarded as a nonlinear programming one. 
The main advantage of this approach (adopted in the works by Keerthi and 
Gilbert and by Mayne and Michalska) is that many well-established nonlinear 
programming techniques are available to solve Problem 1. ii) Off-line computa- 
tion. This approach implies that the control laws i E I have to be 

computed “a priori” and stored in the regulator’s memory. Clearly, the off-line 
computation has advantages and disadvantages that arc opposite to the ones of 
the on-line approach. No on-line computational effort is requested from the reg- 
ulator, but a very large amount of computer memory may be required to store 
the closed-loop control law. 

Now we state the basic stability result for the above RH controller (more 
details can be found in [19]). In this connection the following assumptions are 
needed: 
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(i) The linear system = Ax^ + Buf , obtained via the linearization of the 



system (1) in a neighborhood of the origin (i.e., A = 



A df 



dXf 



and 






df 



Xt = 0 , «£=0 



duf 



), is stabilizable. 

^t=0, M(=0 

The transition cost function h{x,u) depends on both x and u, and there 
exist two strictly increasing functions r, s e , with r(0) = s(0) = 

0 , such that r(||(a:, u)||) C h{x,u) C ,s(|| (a:, u)||), Vx e X, Vu e t/ , where 

(x, u) = col (x, u) . 

There exists a compact set Xq C X, Xq G Z , with the property that 
there exists a control horizon MCI such that there exists a sequence 
of admissible control vectors {u^ G U, i — t, . . . ,t + M 1} that yield an 
admissible state trajectory x^ G X, i = t, t + 1, . . . , t + Af ending in the 
origin of the state space (i.e., Xf+M = O) for any initial state Xf G Xq . 
(iv) The optimal finite-horizon feedback control functions j), j = 

t, . . . , t -h ® 1 , which minimize the i-th cost (3), arc continuous with 
respect to Xj , for any Xj G X and any finite integer X* C 1 . 



(iii) 



Note that assumption (iii) substantially concerns the controllability of 

the nonlinear system (1). Let us now denote by Jpj^{xt,N'‘,a',P'') = 

t+N-l 

h{xf^° + h’pixf'fj^) the cost cor- 

3=t 

responding to the i-th optimal X-stage trajectory starting from X( (to simplify 
the notation, we let hf.{x) = a®||x||pj , without any ambiguity whenever a® and 
P® need not be rendered explicit). Then the following theorem can be proved. 

Theorem l.[19] Let us consider a generic i € I . If assumptions (i) to (iv) are 
verified, there exist a positive scalar d® and a positive-definite symmetric matrix 
pi g such that, for any N'’ C M and any a® G M, a® C d® , the following 

properties hold: 



1 ) The RH control law stabilizes asymptotically the origin, which is an equi- 
librium point of the resulting closed-loop system. 

2) There exists a positive scalar /3® such that the set W(X®,a®,P®) G Z, 

W{N^,a^,P'') = {x G X : Jpyy(x, fV®, a®, P®) C /?®}, is an invariant 
subset of Xq and a domain of attraction for the origin, i.e., for any 
xt G W{N'‘ ,a^ , P') , the state trajectory generated by the RH regulator 
remains entirely contained in W(N'^, a®, P®) and converges to the origin. 

Remark 3. More details on the above theorem can be found in [19]. We want 
only to remark that Theorem 1 is constructive in the sense that, for any X®, 
the scalar d® and the matrix P® can be determined explicitly, thus making the 
design of the RH regulator implemcntablc. 
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4 The interface level 

As already mentioned in the Introduction, the discrete-event level and the 
continuous-state level communicate by means of an interface level made up of 
two distinct parts: a first module implementing the algorithm T> which trans- 
forms the batches of the state and control vectors of the plant into an aggregate 
information vector l( , and a second module simply composed of the set T of 
possible control functions. 

In general, the availability of the batches , . . . , X(} and control vectors 

• 5 ^t} allows to perform very general operations like, for instance, the 
identification of a model of the plant, the condition monitoring of the closed- 
loop system, and so on. In the present preliminary work we consider the very 
simple situation = 0 , but this is not restrictive. 

We now detail a little more the algorithm V . At each time instant t it receives 
as an input the values Xt and ut coming from the CSL. On the basis of these 
values, T> defines the set I{xt) = {z G / : G Xi\ . Of course, it is necessary 

to use here a set of regions since Xt could belong to the intersection of two or 
more regions. Moreover, T> identifies the index it of the control action presently 
applied and the set of feasible control functions for time instant t + 1. Then, in 
addition to I{xt ) , we introduce also a set of feasible control functions J{xf) ■ 
As will be seen in Section 6, this set is built taking into aceount some stability 
criteria. Clearly, in a more general setting, this feasible set could be characterized 
in a different way. The aggregate information vector t( is now defined as 

j/ [xt) = co\[T{xt),it,J{xt)] 

The second module of the IL is very simple since it receives from the DEL 
the index i,i € I, of the control strategy to be applied in the CSL and it sets up 
the CSL regulator with 

5 Discrete— event system model 

The DEL, which constitutes the discrete-event supervisor, is modelled as a de- 
terministic automaton defined as the seven-tuple {E, X , S, ji, 6, 4>, A), where E 
is the automaton alphabet, that is a finite set of symbols in which each symbol 
identihes a specific event, X is the finite state set of the automaton, is the 
feasible (or enabled) event set, defined for all ^ G A with A(^) C E, p,t is the 

automaton input function, nt '■ X {if t — card [X/ (xj)] ), <5 is the au- 

tomaton state transition function, 6 : X C E (8>— > X , (f is the automaton output 
function, : X (8>-^ I, A is the automaton clock structure. The clock structure A 
is defined as A = {xj,j — 1, 2, . . .} defining Tj as the time instant of occurrence 
of the j-th event (of any type). 

The DEL receives as input the aggregate information vector X/ bringing 
information about the CSL as generated by the IL. The automaton input func- 
tion fit transforms the information vector X/ into the corresponding automaton 
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state ^ G X. Then, the dynamic behaviour of the automaton A obeys to the rule 
defined by the state transition function, which means that the next automaton 
state can be expressed as: 

r = 5(^,e) (4) 

where ^ is the automaton current state and e G <S'(0 is the next event occurring. 
To analize the automaton dynamics in a synchronous way, it is necessary to define 
the automaton state in relation to a specific discrete-time instant t. To do this, 
let us define as the automaton state at time instant t. Of course, the time 
instant t is referred to the synchronous clock structure of the CSL and it is not 
necessarily included in the automaton clock structure A. The structure of the 
automaton state is defined as 

where e[(_i is the sequence of events occurring in the time-window [t (8> 1, t) . 
Equation 4 becomes now 

6+1 = <^('?t>e[qt+i)) (5) 

where j+i) denotes the sequence of events occurring in the time-window [t, t+ 
1) . More specifically, we have 

^ I C0 if ^J- Tj G [t,<+ 1) 

e[t,t+i) = S ^ 

y {cs, j C s C , with i , : Tj - , . . . , r,+ G [t, t + 1) otherwise 

where C 0 denotes the null event, which means that no external event has 
occurred in [t, t+1). The state transition function S is defined in a straightforward 

way: by using the compact notation = ^(6e), we have (the symbol C 

denotes the composition) 

6+1 = ^(6,e[t.t+i)) = corc^b- (e,) 

Remark 4. For most applications, we expect that the rate of event-changes is 
much slower than the sampling rate. Equivalently, it is very unlikely that, for 
some t, there exist Tj and r^+i such that t C Tj < Tj+i < t + 1 . 

The DEL objective is that of choosing, depending on the current automaton 
state and on the sequence of events occurred in the last considered time-window, 
the control action to be adopted in the next time interval. To do this, the DEL 
includes a decisional look-up table T in which each combination of a vector 
X/ (x) and of a type of event is associated with the index of the best control 
function to use. This means that each entry of T is constituted by a triple 
{x/ [x),e,i},x G X, e G £^, i G /. It is important to remark here that the best 
control strategy to be adopted at time interval t+1 can be different from the 
one applied in t even if no external event occurs in [t, t+1), but depending only 
on the system state 6- 

Thus, in the time interval (t, t + 1), the state transition function 6 works in 
this way: 
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1 . on the basis of and e[t.t+i) it identifies the index j, j G I of the best control 
strategy to be applied; 

2 . it checks whether j G l( {xt) or not; 

3. in the affirmative case, j is inserted in the next automaton state, otherwise 
the new control strategy is defined as that with index belonging to x/ {xt) 
and with minimum distance from 7 ^, in some specified metric. 

The automaton output function (p defines the correspondence between the 
automaton states and the integer indexes identifying the different control strate- 
gies . Thus, <j) simply realizes the mapping between an automaton state ^ G X 
and the index i G I corresponding to the i-th control strategy 7 ^ . In this way, the 
DEL can change the control strategy to be applied to the plant, in dependence 
of some information about the plant state vector and control vector, and, above 
all, according to the possible occurrence of asynchronous external events. As an 
example, a simple re-configuration scheme in a diagnostic system can be easily 
put in the form just presented. 



6 An invariance property of the hybrid control scheme 



In the present section, an invariance property of the hybrid control scheme is 
addressed. More specifically, it will be shown that all trajectories of the hybrid 
system lie in a suitable invariant set. 

Let us recall Theorem 1 in Section 3 and define the sets tb = 
W(A*; ah P'), i G I. Clearly, we have that 

x,c tel (6) 

Notice that the evolution of the hybrid-control scheme can be described by the 
coupled equations 

it = Ih[I{ {xt)] 

' 6+1 = e[t,t+i)) 

, Xt+i — f[X't, 70({t+i) (^t)] 

Let us now introduce the mapping 



A 



P:XC XC Et 



where 



r c 






if j e : Tj e [t, f + 1 ) 



j^—j +1 times 

y EC CCCC E if 3 , j+ e : Xj- , . . . , Xj+ G 1) 
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In Proposition 1 below, the invarianee property will be stated in terms of the 
mapping T . Before stating and proving the invariance result, some further useful 
quantities and notations should be introduced. 

dij{x) = \\-u{x)( 2 )-ij{x)\\^ Wx e X,Vi,j e I, i j (7) 



A 



AW = ® rpH[f{x,j,{x)),N\a\P^] , Vx e A”,, Vi e I 



( 8 ) 



kl-H = ke 



JW W, N\a\, P*) 0 JW (A, N\ a\ P*) 



C k llx' 0 x" 



Vx',x" G 'i i ^ I (9) 



fc/(x) = fc e M+ : ||/(x, u') 0 /(x, u")|| C fc ||u' 0 m”|| , Vu',u"eU (10) 

Finally, let J{xt) = {j G I : dij{xt) < A,{xt) /k'ppkfixt), Vi G P(xt)} , and 

introduce the set — \^Xi. Clearly, we now have that T{xt) C J7(x() . In 

iei 

this connection, we can write X/ (xj) = col [X(x(), if, J7(x() \X(xj)] , and we have 
that ■i/'t = card(j7(3^t)) + 1 ■ Then, the following result can be proved. 

Proposition 1. First, assume that Theorem 1 holds true Vi G I. Moreover, 
assume that the following contraint is embedded in the DEL: if G , then 
m^ueyt, t+i))) G ; irrespective of the specific values of the arguments 

Ct; C[t,t+i) ■ Then, the set is contr oiled-left-invariant with respect to X C Et 

and the mapping P , that is 

VtCO, X[X+,X,Et]C X+ (11) 



Proof of Proposition 1. Let us consider a generic time-instant t C 0, and 
generic ^ e X, e[t,t+i) G Et ■ Wc need to prove that E{xt, Ct, e[t,t+i)) G 
A+, Vxt G X^ . Two different cases have to be addressed. 

1. Xf G X^ and (f>{d{^tX[t,t+i))) = i G X(Xf) . Thanks to the constraint in the 
statement of the proposition and to the invariance of Xj (see Theorem 1), 
we have X{xt,^t,e[t,t+i)) ^ C X+ . 

2. Xt G X^ and ?!>((^(Ct, e[t,t+i))) ~ j ^ J{xt) \ I[xt ) . In this case, the proof 
is a little more involved. For any i G X(X() , the set Xi is an invariant 

set under the action of the control function y*, and the function Vi(xt) = 
J],.^(xt, a®, P®) is the corresponding Lyapunov function (see [19]). Hence, 

we have: 



® Vi{xt) = ®Ai{xt) < 0 
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Now, by using constraint 2, we have 



C II /(Xj, 7 j(a::()) ® /(as'j , 7 j II C kpp kf(^x^') || 7 j(xj|) ® 7 j(xy) 
C kppkf{xt)dij{xt) < ®Ai{xt) < 0 



Then Vj is a Lyapunov function and Xi C X^ is an invariant set under the 
action of 7 ^, thus proving that T{xt,^tX[t ,t+i))C X+. 



Remark 5. It is worth noting the meaning of the constraint imposed in the 
DEL (see Proposition 1). Actually, the set of indexes J{xt) denotes the control 
functions that, in connection with the present state Xi, do not give rise to control 
actions with too large deviations with respect to those a-priori guaranteeing the 
decrease of some Lyapunov function (see also Theorem 1). 



7 An application of the hybrid controller 

In the present section, a complete simulation example for the proposed hybrid 
receding-horizon controller is presented. First, the description of the system 
dynamics and of the proposed control objective is given. Then, the definition 
of the state automaton realizing the DEL and of the corresponding transition 
function is reported. Finally, significant simulation results are presented and 
discussed with particular emphasis to the comparison between the application 
of the proposed hybrid control scheme and that of a control scheme including 
only the CSL. 



7.1 The considered nonlinear system to be controlled 

The system considered in this example is relevant to a high-performance aircraft 
affected by wing-rock when flying at a high angle-of-attack ([20], p.l80). The 
roll motion of the aircraft is modelled by the equation 

0 = (() + <^ + 101 (12) 

where 0 is the roll angle, 5a is the aileron deflection angle, = ®26.67, {>2 = 
0.76485 , = (g)2.9225 , and b — 1.5 . As we disregard the time constant of the 

first-order actuator dynamics, 5a constitutes the control input (i.e., u = 5 a)- 
After a first-order Euler’s discretization of (12) with respect of the time, we 
obtain 

a:i,t+i = xit + Aix2t 

X 2 X +1 = X 2 t + At [l?! Xit + 'i ?2 X 2 t + 'ds X 2 t fm{xit) + b Ut] 
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where x\t = <p{tAt ) , X 2 t = <p{tAt ) , At = 0.01 , and fm{4‘) = ^ 



In (2 + 



^P<t> 



+e ® ln4 



is a differentiable function that approximates the modulus \4>\ 



for large enough values of the parameter /3 (we took (3 = 50). Moreover, the 
system state is affected by perturbations producing variations of the state x\ 
randomly distributed between (8)0.5 and 0.5. Such perturbations are modelled as 
external events occurring asynchronously and affecting the system behaviour. 

Without feedback control, the origin of the state plane is an unstable equilib- 
rium point; the trajectories starting from initial points located in a neighborhood 
of the origin exhibit cycling oscillations typical for wing rock. The receding- 
horizon cost defined in this case is the following: 



t + N-l 

Jt = [pw? + 400«®u,+i)2] , t = 0,l,... (13) 

i=t 



where Vi — xu , and p is a positive scalar. The addressed control problem is the 
stabilization of the closed-loop system, with = 0,t C 0. The control functions 
included in class T are determined as defined in Section 3 and, in particular, are 
characterized by = 5 and by the fact that the control action is weighted by 
specified control weights pj,j E I. 



7.2 Definition of the state automaton realizing the DEL 

The system stabilization is here sought by exploiting the stabilizing properties of 
the proposed hybrid control scheme. The control objective here addressed can be 
also achieved by applying only one of the control functions included in F. The 
necessity of adopting a hybrid control scheme arises when observing that the 
control scheme can be further improved by adding a discrete-event supervisor 
capable of choosing the best control function to be applied among the defined set. 
Actually, the feedback control scheme realized by using only the CSL exhibits 
improvable behaviours particularly in correspondence of; 

i) large values of the state variable X 2 ; in this case, the regulator should perform 
in a slower way, thus, requiring a higher weighting factor for the control 
action; 

ii) small values of the state variable X 2 ; analogously, now, the control action 
could be faster, that is, the weighting factor for the control action should be 
lower. 

The objective of the discrete-event supervisor has been identified, in this 
particular case, with the choice of the best weighting factor for the control action, 
depending on the system state and on the possible occurrence of perturbations. 

The action implemented by the discrete-event supervisor is, in general, that 
of choosing a control function with higher p in case (i) and a control function 
with lower p in case (ii). 

A large experimental campaign has led to a better definition of the DEL 
action. More specifically, it has been found that: 
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— the increase (decrease) of the weighting factor of the control action between 

two control functions subsequently applied, namely G I, i ^ j, is 

identified with a predefined value p, that is, 

Pj — Pi C p {pj — pij p) 

(a suitable choice is p = 10.0). 

— the weighting factor of the control action should be increased also when a 
change of sign occurs between two subsequent values of the state variable 

— if the weighting factor of the control action has been increased (decreased) 
in the previous time interval, it should not be varied; 

— if a perturbation in the system state has occurred, the weighting factor 
should be reset to its default value (= 100.0). 

Following these reasoning lines, the state automaton realizing the DEL has 
been defined in the following way: 

— E ^ {eo, ei}, where cq = null event, ei = perturbation on the system state; 

— X = = col[^^,^^]: G {®1, 0, 1, 2}, G {a,b,c}}. The first component 

of the automaton state, namely identifies the change occurred on the 
weighting factor of the control action in the preceding time interval, whereas 
the second component of the automaton state, indicates whether the 
present system state satisfies specific conditions or not; 

— of course, in this case, X(^) — E,\/^ G X. 

More specifically, the first component of the automaton state vector at 
time instant t takes on the following values: 

- = ®1, which means that pj has been decreased at time instant t: 

- = 0, which means that pj has not been changed at time instant t\ 

- = +1, which means that pj has been increased at time instant t; 

- — +2, which means that pj has been assigned to its default value {pj = 
100.0) at time instant t. 

As regards the second component of the automaton state vector, it can take 
the following values and meanings: 

- — a, which implies that the value of the state variable X 2 is greater than a 
predefined upper bound and a change of sign has occurred between the two 
last subsequent values of X 2 ; 

- ^1 = 6, which implies that the value of the state variable X 2 is smaller than 
a predefined bound and no change of sign has occurred between the two last 
subsequent values of X 2 ; 

- = c, which implies that the value of the state variable X 2 is between its 
lower and upper bounds and no change of sign has occurred between the two 
last subsequent values of X 2 - 
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7.3 The automaton transition function 

The automaton transition function defining the dynamic behaviour of the state 
automaton can be expressed as follows: 

IF C ei THEN 

6 + 1 = [2,^2]; Ve? 

ELSE 

IF cl = +1 OR cl = THEN 

6+1 = [0,6'] V6' 

ELSE 

IF cl = a THEN 

6+1 = [l,a] 

IF cl = b THEN 
6+1 = i'S)l,b] 

IF cl = c THEN 

6+1 = [0, c] 

The automaton transition function acts by changing the value of 6 in de- 
pendence of the previous values of and C^, and of the possible occurrenee of 
an event of type ei in the preceding time interval, and according to the rules 
above defined. 

The automaton output function updates the value of the weighting factor of 
the control action and chooses, as the next regulator to use, the RH regulator 
characterized by such a factor. 
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7.4 Simulation results 

Some numerical results are here proposed relevant to the comparison of the be- 
haviour of the system with application of the CSL and of the hybrid control 
scheme. The control problem here addressed is the stabilization of the system 
described in Subsection 7.1. In the considered example the initial value of the 
system state is xio = 0.5 and X 20 = 0.0. Moreover, asynchronous perturbations 
affect the system by modifying the value of the state variable xi with variations 
randomly distributed between (g)0.5 and 0.5. The perturbations have been mod- 
elled by means of the event ei whose occurrence time is again a random variable 
uniformly distributed between 30 and 70 time intervals. 

The set F of optimal feedback control functions, with reference to the 
receding-horizon cost given in (13), is determined as in Section 3. Such functions 
are characterized by Af = 5 and, above all, are parameterized by the value of the 
control weight p. Being the aim of this section to compare the performance of the 
hybrid control scheme with that of an optimal feedback regulator acting only at 
the continuous level, it has been necessary to identify the best control function 
to be applied to the plant in the CSL. Such a function has been determined as 
7 j,j G I,Pj — 100.0, and 100.0 is also taken as the default value of the control 
weight to be used by the discrete-event supervisor. 

The results of the above mentioned comparison between the overall hybrid 
control scheme and that of the CSL only are compared in Figs. 2—4. The dotted 
lines are referred to the behaviour of the state variables Xi and X 2 by apply- 
ing only the RH regulator characterized by pj = 100.0 whereas the solid lines 
correspond to the application of the hybrid control scheme. 



(a) (b) 




Fig. 2. (a) Behaviour of x\ under the action of the hybrid controller (solid line) 
and under the action of the RH controller (dotted line), (b) Zoom of the same 
trajectories in the time interval [100,220]. 



As can be noticed from Figs. 2 and 3, the transient behaviors of the state 
components are significantly different in the two cases. More specifically, in most 
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(a) (b) 




Fig. 3. Behaviour of X 2 under the action of the hybrid controller (solid line) 
and under the action of the RH controller (dotted line), (b) Zoom of the same 
trajectories in the time interval [100,220]. 



cases the behavior of x\ is faster and with a smaller overshoot. Moreover, the 
interesting feature is that no trial-and-error procedure should be devised a- 
priori in order to find the best control weight, in the sense that the discrete- 
event supervisor is able to choose it on-line with respect to a given prc-spccificd 
criterion. In order to appreciate the effects of the control weight changes, in Fig. 
4 the behavior of the control weight is plotted together with a zoom of the two 
behaviors of x\. Notice that the better transient is obtained at the price of some 
lack of smoothness of the state behavior. However, this effect is not surprising 
as hybrid systems like the one considered in the paper are in general described 
by discontinuous state equations [11]. 



8 Conclusions 



The design of an hybrid control scheme for non-linear discrete— time systems is 
the objective of this paper. The proposed control scheme is made of a continuous 
level and a discrete-event supervisor. At the continuous level a set of optimal 
feedback control functions is determined, whereas the discrete-event level realizes 
the choice of the best control function, in the defined set, to be applied in each 
time interval. 

The effectiveness of the proposed approach is first investigated by proving a 
preliminary result about the stability of the overall scheme, and, then, shown by 
means of a complete example. The stability analysis set up here goes towards the 
possibility of embedding the considered hybrid system into a switched system in 
order to exploit the same reasoning lines as in [11]. To this end, the definition of 
invariant sets for the considered systems and some stability concepts regarding 
the same invariants are already established and reported. 
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Fig. 4. Zoom of the behaviour of x\ under the action of the hybrid controller 
(solid bold line) and under the action of the RH controller (dotted line). The 
behaviour of the control weight is also depicted (solid line). The values of .xi and 
of the control weight, are reported on the left and on the right axes, respectively. 
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Abstract. A standard problem in hybrid control systems theory is to 
design discrete, or symbolic, feedback for a given continuous plant. When 
specihcations are discrete, this problem can be solved by hrst approxi- 
mating the continuous plant model by a (nondeterministic) automaton, 
and then synthesizing discrete (supervisory) control for the automaton. 
A necessary condition is that the approximation behaviour contains the 
behaviour of the underlying continuous plant model. Then, any controller 
forcing the approximation to obey the specihcations will also force the 
continuous model to satisfy the specihcations. We use a version of this 
approach which allows adjustment of approximation accuracy to address 
two simple process control problems: supervisory control of a three-tank 
laboratory experiment and safety enforcement for an evaporator. In both 
cases, the entire design process is carried through: we hrst determine a 
suitable abstraction, compute the minimally restrictive supervisor, and 
then present examples for closed loop trajectories. 



1 Introduction 

This contribution deals with an approximation based approach to hybrid control 
systems design and its application to two specific examples - supervisory control 
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of a three-tank laboratory experiment and safety enforcement for an evaporator. 
This paper builds on previous work by the first and third author [11,12,13]. Its 
purpose is twofold: on the theoretical side, the accuracy of the approximation 
step is increased by “trimming” the transition structure suggested in [12,13]; this 
improvement follows directly from the input /output view adopted in the paper. 
On the practical side, it is demonstrated that approximation based approaches 
are indeed feasible for a class of small but realistic hybrid control problems. 

Using discrete approximations, or abstractions, to design discrete feedback 
for continuous plants is an appealing idea: such an approximation based approach 
“translates” potentially difficult hybrid problems into purely discrete problems, 
which can subsequently be solved using tools from DES (Discrete Event Systems) 
theory. The approach constitutes a rigorous design method, if the approxima- 
tion behaviour (the set of all pairs of input/output signals that are compatible 
with the approximation equations) contains the behaviour of the “underlying” 
continuous plant model. Then, any controller that solves the problem on the 
discrete level (forces the approximation to obey a given set of specifications) will 
also force the continuous plant model to satisfy the specifications. 

Approximating continuous systems by discrete models for the purpose of feed- 
back design has been suggested by Antsaklis, Lemmon, and Stiver (e.g. [17,1]), 
and by Lunze (e.g. [9]). In the original version of their approach, approximation 
accuracy is completely determined by measurement quantization; the measure- 
ment map Qy : K” ^ Yd (where Yd is a finite set) partitions the plant state 
space K", and the “cells” induced by this partition correspond to the abstrac- 
tion states. However, approximation accuracy is a crucial parameter for con- 
trol systems design: if the discrete approximation is “too coarse”, no suitable 
controller exists, and the problem cannot be solved unless one can change the 
granularity of measurement quantization (this phenomenon will be illustrated 
for the three-tank example). Hence, one needs an abstraction scheme which, for 
a given measurement map, allows approximation accuracy to be adjusted to var- 
ious specifications. Such a scheme has been suggested in [11,12,13]: it provides 
a totally ordered set of discrete abstractions, where ordering is in the sense of 
approximation accuracy. The approximation step in this scheme can be further 
improved by adopting an input /output point of view as described in [14]. 

This paper is organized as follows: in Section 2, it is explained how to build 
discrete abstractions for a continuous model by first approximating its behaviour 
and then generating realizations for the approximate behaviours. Section 3 sum- 
marizes a procedure for synthesizing minimally restrictive supervisors for the 
discrete abstractions. This synthesis method is a modification of Ramadge’s and 
Wonham’s supervisory control theory (e.g. [15,16]); it can be directly applied to 
the discrete models obtained in the approximation step. It has been described 
in [13]; we include a short summary to make the present paper self-contained. In 
Sections 4 and 5, we use our approach to design supervisory control for a three- 
tank laboratory experiment and a safety enforcement scheme for an evaporator. 

Finally, a remark concerning notation and terminology: signals are interpret- 
ed as maps; the domain of a signal is time (denoted by T), its codomain is 
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referred to as the “signal space”. Time can be continuous (T = K+) or discrete 
(T = {tQ.ti , As most of our paper is set in a discrete-time framework, 
we use the same symbol for signals in continuous and in discrete time. The 
important distinction is between continuous-valued and discrete-valued signals: 
the codomains of the former are dense subsets of Euclidean space, the latter 
“live” (take values) in discrete sets. Signals are represented by lower case letters, 
their codomains by the corresponding upper case letters. Discrete- valued signal- 
s are characterized by the subscript “d”. For example, yd ■ Yd 

is a discrete-valued measurement signal which is defined on the sampling grid 
{to, ti, ■ ■ ■ } and “lives” in the (discrete) set Yd- The codomains of all discrete- 
valued signals are assumed to be finite, their elements (the possible values the 
signal can take at each instant of time) are characterized by superscripts: the 
z-the element in the set Yd, for example, is denoted hy yy . Finally, irrespective 
of the nature of time, we call a plant model “continuous” (“discrete”), if its state 
variable is continuous- valued (discrete- valued). 



2 A Hierarchy of Discrete Abstractions for a Given 
Continuous System 

In this section, we introduce an ordered set of discrete abstractions for a given 
continuous plant model. We will first describe the continuous model, then specify 
the behaviours that the discrete abstractions are required to exhibit; then, in a 
subsequent step, we will come up with minimal realizations for these behaviours. 



2.1 The Continuous Plant Model 

The continuous “base” model of the plant is described by the (vector) differential 
equation 



-{t)=f{x{t),Ud{t)), (1) 

where t G K+ denotes time and x{t) ^ X C K" is the continuous state at time t. 
Ud{t) G Ud is the control input, with Ud being a finite set of symbols: 

Ud = {Ud\--- > 4 “^}- 

We work in a discrete-time framework with fixed sampling grid T = {to, t\,t2, 

. . .} C K+. Hence, the control signal is piecewise constant and can only change 
value for t = k = 0,1, . . . . Likewise, measurement information is only avail- 
able at the sampling instants. Hence, we switch to a discrete-time version of the 
continuous plant model. 



X{tk+l) = f{x{tk),Ud{tk)) 

Vdi^k) 



( 2 ) 

(3) 
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where f{x{tk), Ud{tk)) is the solution of (1) at time t^+i for initial condition x(tfc) 
and control input Ud{t) — Ud{tk), tk C t < tk+i (the solution f{x, u,i) is assumed 
to exist for all x G X and Ud G Ud)- The measurement signal yd ■ T ^ Yd “lives” 
on a finite set of symbols 

^ci = {Vd'’^--- 

f : XC Ud ^ X is the state transition map, qy : X ^ Yd the measurement map. 
Without loss of generality, the latter is required to be onto. 

2.2 Behaviours 

Let T = {to, ti, . . .} be the chosen sampling grid. Denote the set of all functions 
from T into {UdC Yd) by {UdC Yd)^ ■ Then, the (discrete-time) behaviour of the 
continuous plant model. Be C {Ud C Yd)^ , is the set of all pairs of control and 
measurement signals which are compatible with the model equations (2), (3). 
For a survey on “behavioural” systems theory see [19]. 

Clearly, a necessary condition for any discrete abstraction is that its be- 
haviour Bi must contain the discrete-time behaviour Be of the underlying con- 
tinuous model: 



Be c Bi (4) 

implies that every sequence of input /measurement symbols that the continuous 
plant model can generate, can also be produced by the discrete approximation. 
If this condition were violated, the continuous system could respond to a given 
input signal with an unacceptable measurement signal which would not be pre- 
dictable by the discrete approximation. Hence, this unacceptable phenomenon 
could not be suppressed by a control strategy based on the discrete approxima- 
tion - the approximation would be useless as far as control systems design is 
concerned. As illustrated in Figure 1, this “abstraction condition” is also suffi- 
cient for the purposes of control systems design: 

suppose we design a causal (discrete) feedback controller (with input signal 
yd and output signal Ud) and hook it up to the discrete abstraction. Obviously, 
the controller behaviour Beontroiier is also a subset of {UdC Yd)'^; the feedback 
loop consisting of controller and abstraction exhibits behaviour Bi H Beontroiier 
— only those pairs of input /output signals “survive” that are compatible with 
both abstraction and controller equations. From Figure 1, it is immediately clear 
that the “abstraction condition” (4) implies: 



B,nB. 



( 5 ) 



in other words: if the controller forces the discrete approximation to obey the 
specifications, the feedback loop consisting of discrete controller and continuous 
“base” model will also meet the specifications. 

It is worth while pointing out that “blocking” {Be H Beontroiier = 0 ) cannot 
occur within the proposed framework: as we work on a fixed sampling grid T, 
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B, 



'controller 



Fig. 1. Visualization of “abstraction condition” . 



“blocking” would imply that connecting controller and continuous plant mod- 
el would, literally, stop time. In mathematical terms, “blocking” cannot occur 
within our framework, as any (discrete-time) feedback loop composed of a strict- 
ly causal system (the continuous plant model (2), (3)) and a causal system (the 
controller) has a solution trajectory on T. 

If (4) holds, the “size” of the difference Bi \ Be is an indicator for the accu- 
racy of the discrete approximation: the “smaller” Bi \ Be, the smaller the loss 
in “prediction power” when replacing the continuous model by its discrete ab- 
straction. The trivial abstraction (characterized by the fact that, regardless of 
previous control inputs, every measurement symbol can occur at each sampling 
instant) has no “prediction power” whatsoever. It is obvious, that no controller 
can enforce the specifications on the abstraction level, if the plant approximation 
is “too coarse” (if, for example, the trivial abstraction is chosen). 



2.3 Abstraction Behaviours 



In order to specify desired abstraction behaviours, we need a bit of additional 
notation (sorry): recall that T — is the sampling grid. Denote the 

intervals {to, • ■ • , tk} and jtfe+i, . . . } by and Ffc+j respectively. If tk refers to 
the present sampling instant, comprises “the future” , and Tfe “the past and 
present”. Dehne C {Ud C YdY’^ to be the restriction of the behaviour Be to 
the interval Tk'- 



B’^ := (6'= e {Ud C Ydf'‘ I 3b’^+ e {Ud C Ydf^+ such that [6^ e Be} ■ 

Finally, introduce the predicted output set 3^c(&^) as the set of all measurement 
symbols that the continuous model (2), (3) can possibly generate at time tk+i 
if the string 6^ has occurred. Then, the continuous system behaviour can be 
written iteratively as : 












^yc{b^)}- 



( 6 ) 
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Now, we define less accurate prediction sets I — 0, 1, , by using only 
“recent data”. By “recent data” (denoted by 6^’*), we mean the restriction of 
to the interval 

rp f {ik — lj • ■ ■ ; if ^ ^5 

,tk} a kc 1. 

Hence, represents a string of control and measurement symbols reaching 
back to time tk-i or, if A: C I, to the initial sampling instant to- Obviously, 

-fe _ / ii k>l, 

^ ^ \ 6'=’' if A: C b 



yi{b^’^) is defined as the set of all measurement symbols that the continuous 
model (2), (3) can generate at time tk+i if the string b^’^ has occurred during 
the time interval Tk,i- Clearly, 



/ ydb'^) if fc > ^ 

\ ye{bd like I 



i.e. yiibf^d is obtained as the union of the prediction sets of all data strings 
b^ which coincide on the interval Tk,i (but may differ during the “distant past” 
Tfc_;_i) (see Figure 2 for an illustration). Therefore, 

yQ(d’^) c yiid’d c ■ ..yid^d ...c y^hd- (7) 




Fig. 2. Predicted output sets. 



From the prediction sets T; , the approximation behaviours Bi can be defined 
iteratively by 



:= B 



0 
c ’ 



: = 




(8) 
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From (6), (7), (8), and (9), it follows immediately that 

BoU BiU ...Bi...a Be. (10) 

Now that wo have specified the abstraction behaviours Bi, I = 0, 1, , we 
only need to explain how to find discrete realizations (state models) that gener- 
ate these behaviours, or, equivalently, the corresponding output prediction sets 



2.4 Realization of Abstraction Behaviour Bi 

In this section we show how to build a discrete state model A[ with predicted 
output set and, consequently, behaviour Bi. We first select the state set 

of Ai. 

Recall that, by definition, the state of any dynamical system summarizes all 
the information that, together with the current input, is needed to predict the 
future: Clearly, to compute the output prediction set yi{b^’^)^ we need to know 
Hence, we define the state Xd of the abstraction Ai at time tk to consist of 
all the control and measurement symbols in the string 6^’^, with the exception 
of the current control input Ud{tk)- For I — 0, the result is trivial: 

^d(^fc) • — Vdibk)] 

for ? = 1, 2, . . . , we get: 

f [yd{to)] , A k = 0, 

d^d{bk) • — \ ([^^(^ 0)5 • • • ; yd(^fc)] ; [^d(^o); ■ ■ ■ i l)]) ? if A,' 1, . . . , /, 

( i[yd{tk-i), ■ • • , yd{tk )] , [ud{tk-i), ■■■ , ud[tk-i)]) , A k> 1. 

(11) 

This choice is reminiscent of observer canonical realizations in (continuous) con- 
trol theory (e.g. [8]) - the state is made up of known quantities, and hence is 
trivially observable. As both Ud, the set of control symbols, and Yd, the set of 
measurement symbols, are finite, the state set Xd of the discrete abstraction is 
also finite: an upper bound for the number of elements of Xd is given by the num- 
ber of different strings (11) one gets by exhaustive permutation of measurement 
and control symbols. 

It is obvious, however, that the continuous system (2) - (3) cannot gen- 
erate all these strings. In the next step, one therefore introduces a “feasibil- 
ity check” and eliminates all strings which are not compatible with the con- 
tinuous model (2), (3): let 1 D p □ I and denote /(x,u^*^) by fi{x). Then, 
( y^d’"’’^' • ■ ■ > ’ '^d'‘'’^’ ■ • • I element in the state set Xd of 
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the discrete abstraction if and only if there exists a solution x G X for 

= % {fik-i {hk-2 (• • • ■■■))) 



(12) 

(13) 



= <ly 

Vd" = % (^) ■ 



(14) 

(15) 



This is a formal way of saying that there exists a value x of the continuous state 



of control symbol- 
of 



Uk~p) 

Vd 






(ik-p) 

d ’ 



> “d 



variable, such that applying the string 

s makes the continuous model respond with the string 
measurement symbols. 

This answers the question whether a string , ■ ■ ■ ,ylT‘'l , \u 

is an element in the state set of the discrete abstraction for p = 

1, . . . ,1. What about the case p = 0, i.c. strings of the form y^J'^ ? As we do 
not assume any a-priori knowledge on the continuous system state, it could be 
anywhere in X at time to (when we “start looking at the system”). Hence, any 
measurement symbol in can occur at time to and := Yj, is the set of 
possible initial states of the abstraction. 

Remark 1. It is clear that, in general, feasibility does not have to be checked 
for every combination (11) of measurement and control symbols: if (12) - (15) 



does not have a solution x for a given string 
1, 0 < p < Z, any “extension” 



( 




„,Uk) 
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Vd 


■ ^Vd 
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(ifc-i) 
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,,Uk-\) 


(ifc— p) 


„,Uk) 






{'i'k — p) 




Vd 


■■Vd 


■ ^Vd 




Ud 




• ^^d 



(ik-p) 



p < X ^l, 

J / 

is also non-feasible. Checking the set (12) - (15) for the existence of solutions is 
of course non-trivial, and, in general, one has to resort to an iterative scheme. 
If the transition map / of the underlying continuous model is not known (the 
model is given in continuous-time), each iteration step also needs to call an 
ODE-solver. This might seem to be an awful lot of computation. One has to 
keep in mind, however, that a fairly coarse approximation Ai may be sufficient 
to solve a given control problem. Indeed, for the examples in Section 4 and 5, 
abstraction Ai proves to be adequate. Moreover, for an important special case, 
the feasibility check reduces to a numerically straightforward procedure, namely 
checking whether a set of linear inequalities possesses a non-empty solution set. 
This special case is characterized by the fact that the right hand side of (2) is 
affine in the state x{tk), and the measurement map has the form Qy — Qy —Cy 
(where Cy : M" ^ is a linear map, and the “quantizer” Qy \ W ^ Yd 
partitions into finitely many rectangular boxes with edges parallel to the 
coordinate axes). The computational procedure for this case is described in [12]. 
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This finishes the construction of the abstraction state set Xd- The number 
of elements in X^, is denoted by N^: 

Xd~ (16) 



Up to now, we have only defined the state set of the abstraction Ai. We now 
describe the transition structure. 

Denote the strings of control and measurement symbols associated with a 
particular by u*{x^^'^) and respectively, and introduce a “forget- 

ting operator” X which deletes the “oldest” symbol from strings y’^{xd{tk)) and 

Ud{xd{tk)), if k 



^{u*d{xd{tk))) ■■= I 



[yd{to ), . . . , yd{tk )] , if fc = 0, 1, . . . , I (g) 1 
[yd{tk-i+i), • ■ ■ , yd{tk)] , lik 

[ud{to ), . . . , Ud{tk-i )\ , if fc = 0, 1, . . . ,1(^1 

— Z+l)i • ■ ■ 5 5 if k 



Now, writing down the transition structure of the discrete abstraction is easy - 
at least conceptually: , x^J ^^ ) is a transition in A; if and only if 

1. there exists a £ Yd such that 






'ny*d(4^)), y^r^ 



2 . 



(m) 

Vd 



Vdi^n 

tern model (2) 






, 0 ) 



(3). 



is compatible with the continuous sys- 



The first condition can be verified by simple visual inspection of the abstraction 
states x^^ and x^J^^ : do they “overlap” in the indicated manner? The second con- 
dition can be checked by setting up a set of equations as in (12) - (15) and testing 
it for the existence of solutions. Suppose both conditions hold, i.e. (x[j*^ , , x^^^) 

turns out to be a transition in Ai. Then, and are called the exit state 
and the entrance state of the transition; the control symbol is its transition 
label. Each state x^f’ has an associated unique (measured) output, which is sim- 
ply the rightmost symbol in y*{x^^'^). Hence, for each nonnegative integer I, we 
get a finite Moore automaton as a discrete abstraction for the continuous plant 
model. It is clear that for a given state x|,*^ and a given control symbol u"P , more 
than one y^^'^ £ Yd may satisfy conditions 1 and 2: then, applying at state 
x^^ may drive the abstraction into more than one successor state (each carrying 
a different output symbol) - the resulting automaton is nondeterministic. By 
construction, the resulting state model is a realization of the output prediction 
set Yi . 
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Remark 2. In [11,12], we proposed slightly different abstractions Af. both A; 
and Al share the same state set, but the transition structure of A; is more 
conservative: {x^^\ ) is declared to be a transition of A(, if condition 1 

holds. The omission of condition 2 gives less accurate approximation behaviours, 
but evidently does not affect their ordering. It can be argued that trimming 
the transition structure by adding condition 2 is a direct consequence of the 
input/output view adopted in [14] and in the present paper: deriving the state 
model A; as a realization of the output prediction set yi naturally leads to 
conditions 1 and 2. 

Remark 3. It is understood that the discrete abstractions evolve on the same 
sampling grid as the underlying continuous system: the time needed for a tran- 
sition is the sampling interval ® ti. This explains why the notion of time is 
trivially retained at the abstraction level. 

Remark 4- As the abstraction state Xd{tk) is made up of known quantities (see 
equation (11)), A; is guaranteed to be observable. It is also straightforward 
to prove that every element in Xd can be reached from an initial state Xd G 
Xdo (see [14] for details). Hence, Aj is a minimal realization of the abstraction 
behaviour Bi. 

3 Supervisory Control for a Discrete Abstraction 

For the purposes of control systems design, we convert the Moore automaton A( 
into an equivalent^ transition structure without state outputs. This is done by 
adding to each state a selfloop that is labelled with the appropriate measurement 
symbol yj . To ensure “correct” ordering of transitions, we compose this transi- 
tion structure with the simple clock process shown in Figure 3. This also makes 
time explicit — the tick event represents the passage of one sampling interval 
and can be thought of as being synchronized with 
an external clock. The arrows labelled by Ud and 
Yd represent the occurrence of any symbol from Ud 
or Yd, respectively. The resulting nondeterministic 
transition structure M is referred to as a Discrete- 
time Discrete Event Process (DTP). It includes ex- 
plicit timing (tick-events) and a control mechanism 
(transitions labelled by a control symbol can be dis- 
abled). Its behaviour is defined as follows: count the 
tick-events; denote transitions occuring before the 
first tick by Ud(to), J/d(Io), transitions occuring be- 
tween the first and the second tick by Ud{ti),yd{ti) 
etc.; in this way, the DTP generates a behaviour 
identical to the one of the underlying Moore au- 
tomaton. 

^ in the sense of having the same behaviour as the Moore automaton. 



Yd 

K) 




Ud 



0 



Fig. 3. Clock process. 
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We adopt a supervisory control philosophy [15] to suitably modify the be- 
haviour of the discrete plant model. We use a state-transition-based framework 
which is well suited to deal with nondeterminism in the discrete plant model, 
and which can also handle time^. 

The departure point is a discrete approximation of the plant model in the 
form of a DTP. The mechanism of control is via a subset of transitions that can 
be disabled (prevented from happening). These are the transitions labelled by a 
control symbol. Specifications are also formulated as DTPs, which may or may 
not (depending on whether time is important in the specifications) contain the 
tick-symbol. Forming the parallel composition P = M \\ S oi the plant DTP, M, 
and the specification DTP, S, formally removes all transitions which violate the 
specifications - but this is done without caring for realizability: for example, a 
transition can only be eliminated if it is labelled by a control symbol; a transition 
cannot be eliminated if this implies that the process can reach a state where no 
further tick-event can be executed — stopping time is impossible. The optimal 
supervisor’s job can then be thought of as implementing the “least restrictive” , 
but realizable substructure of P. This is formalized in the following paragraph: 

Let Q and S be the (finite) sets of states and event (or transition) labels of 
P. A transition is represented by a triplet {qc, cr, g.;), with qi 6 Q and a € S. 
qe and q^ are called exit and entrance state; a is the event label. Transitions 
(gel I CTi, qn) and (ge 2 , ct 2 , g^ 2 ) are called partners, if gei = ge 2 and cti = ct 2 
(they have the same exit state and event label). A DTP can be represented by 
a pair (Z\, Qq) where A and Qo are the (finite) sets of transitions and initial 
states, respectively. 

Definition 1. Let P — {A, Qq) be a DTP. The DTP P modeled by the pair 
{A, Qo) is called a Discrete-Time Discrete Event SubProcess (DSP) of P (de- 
noted by P P), if A A, Qq ^Qq, and a transition 5 G A can only be an 
element in A, if all its partners are also contained in A. 

A state g 2 G Q is reachable from a state qi G Q (or, equivalently, gi is 
coreachable from g 2 ), if there is a sequence of transitions from A connecting gi 
with g 2 . A DTP P is called reachable if every element of its state set is reachable 
from an initial state. 

Definition 2. Let P = M \\ S and Ud be the set of transition labels of M (and 
hence P) which can be disabled by a control agent. Let P — {A, Qq) be a reachable 
DSP of P with state set Q and with Qq — Qq. Then P is said to be controllable 
w.r.t. to M, if {qMe, <x, qMi) G Am (the transition set of M), {qMcQSe) G Q, 
and {{qMe, gse), cr, 0) ^ A implies that a GUd (0 means “don’t care”). 

Clearly, a DSP P oi P can only be realized by a controller if it is controllable 
w.r.t. M . Another condition for realizability is that the progress of time can 
never be stopped: 

^ An extension of [15] to (deterministic) DES with timing features is described in [3]. 
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Definition 3. A state qi & Q is said to he tick- coreachahle if there exists a 
state (72 G Q where a tick-transition can occur and which is reachable from q\. 
The DTP P is called tick- coreachable or temporally nonblocking, if every state 
in Q is tick-coreachable. 

Let Pi — , Qq) and P2 ^ Qq) be two DSPs of P. Then the union 

of Pi and P 2 , denoted by Pi U P2 is a transition structure represented by the 
pair {A^ U A'^, Qq U Qq). It is immediately clear that Pi U P2 is another DSP 
of P. The relation ^induces a partial ordering on the set of all DSPs of P. Let 
{Pcn} he the set of all DSPs of P which arc controllable w.r.t M and temporally 
nonblocking. {Pcn} is closed under union. Hence, if non-empty, {Pcn} forms 
an upper-semilattice (with the join operation being U). Clearly, {Pcn} is finite. 
Therefore, the following holds: 

Property 1. If {Pcn} is non-empty, there exists a (unique) greatest DSP of 
P (w.r.t. the ordering via which is controllable w.r.t. M and temporally 
nonblocking. 

If {Pcn} is non-empty, denote its supremal element by Pg. It can be inter- 
preted as the transition structure of P that survives under the least restrictive 
realizable supervisory control policy which guarantees the specifications to be 
met. Ps (and hence the least restrictive control strategy) can be formally synthe- 
sized via a fixed-point algorithm in a computer-aided design environment. This 
procedure has been implemented in C+-|- with an object oriented architecture. 

If {Pcn} is empty, the supervisory control problem has no solution. This 
implies that either the approximating automaton Ai is too coarse (we need to 
provide a finer approximation Ai+i, i ^1), or the specifications are too strict 
(they cannot be met no matter how accurate our approximation is) and need to 
be relaxed. 

Remark 5. “Forbidden state” type specifications are particularly straightforward 
to deal with: they are static, hence we don’t need a specification automaton. For- 
mally, the problem can be solved as follows: certain states in the plant automaton 
Al (and hence the DTP M) are declared to be forbidden. If a DSP of M does not 
contain any forbidden states, it is called safe. Then we simply check whether the 
set of all safe, controllable, and temporally nonblocking DSPs of M is non-empty. 
If so, the problem is solvable, and the supremal element in the set, denoted by 
Ms, is the minimally restrictive supervisory control policy. 

4 A Three-tank Laboratory Experiment 

4.1 Plant and Continuous Model 

Our first example is a three-tank laboratory experiment (Figure 4) which can 
be purchased from amira GmbH, Duisburg. It consists of three plexiglass tanks 
with identical cross sectional area A = 154cm^. The tanks are connected by 
two pipes; tank 3 is also equipped with an outlet pipe. All pipes have cross 
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pump 1 





tank 1 



tank 2 tank 3 



pump 2 



Qo 



Fig. 4. Thrcc-tank laboratory experiment. 



sectional area a = O.Scm^. The pumps attached to tank 1 and 3 can be operated 
independently. Their flow rates are denoted by qi and Q2, respectively. In both 
cases, the maximum flow rate is gmax = Denote the water level in tank i 

hy Xi, i — 1, ... ,3. Now, we can derive a continuous model by writing down 
(volume) balance equations for each tank: 

gi ® gi2 

gi2 ® Q23 

Q2 + 923 <8> 9o • 

Applying Toricelli’s law yields: 

_ f ® ^2 if Xi ~^X2 

( ®azi^a-^flg^/x2 ® x\ if x\ < X2 

az2say^^X2 ® Xs if X2 -^X3 
®az2.2,a\f^\Jx3 ® X2 if X2 < 

Qo = , 

where g — 9.8l|r is the gravitational constant, 0 ^ 23 , and Oz ^2 are loss 
coefficients of the respective pipes. They depend on the geometry of the pipes 
and the water levels in the tanks. In our experimental set-up, the influence of 
water levels on these coefficients was found to be fairly weak; they were therefore 
approximated by constants: 

^Zo ~ ^Z23 ~ ^Z\2 ~ 0.75 . 

We choose a sampling interval of At := tkyi ® tk ~ 100 s and operate the 
system by switching both pumps between maximum flow rate {qi = gmax); half 




(17) 

(18) 

(19) 

( 20 ) 

(21) 

(22) 



A 



A 



A 



dxi 

dt 

dx2 

dt 

dx3 

dt 
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the maximum flow rate — O.hgrjfjax), and “off”. Hence, we get 9 different values 
for the control input Ud (Table 1). We restrict ourselves to highly quantized 



Table 1. Control symbols for the three-tank experiment. 




measurement information: at each sampling instant, we only “see” whether the 
water level in tank 3 is below 30 cm, above 60 cm, or in between: 

if 0 < X 3 ^30 cm 

if 30cm < X 3 ^60cm (24) 

yP if 60 cm < ^Xmax ■ 



4.2 Specifications 

We investigate two sets of specifications: the first one basically requires that the 
water level in tank 3 should, after a short transitional period, always be kept 
in the middle range. An automaton that expresses this specification is shown 

in Figure 5. At the first three sampling instants, any measurement symbol is 

( 2 ) 

allowed. After that, only may occur. There are no restrictions on the control 
symbols, and explicit timing (the inclusion of tick-symbols) is not required. 
Hence, the set of transition labels is Ya- 



O- 

Ri) 

Xg 



Yd 



-o- 



Yd 



-o- 

^,(3) 



Yd 



Ai) 

d,g 



Fig. 5. Specification automaton Sa- 



A slightly more demanding specification is the following. After a start-up 
period of three sampling intervals (where we allow any measurement symbol), 
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( 2 ) 

has to occur at the next ten sampling instants. Then, after a transitional 
period of two sampling intervals, we want the water level in tank 3 to drop below 
30 cm (the symbol has to occur once), before the cycle starts again. This 
specification is captured in the automaton shown in Figure 6. Again, the set of 
transition labels in the specification automaton is Y^. 




Fig. 6. Specification automaton Si,. 



4.3 Discrete Abstractions and Control 

With the algorithm suggested in Section 2, we can now compute the abstractions 
Ai, I — 0,1, .. . . Recall that Aq is the coarsest element in our set of approxima- 
tions. The Moore automaton Aq consists of only three states - the measurement 
symbols , yp\ and yp^ - and it turns out to have 64 transitions. We now ap- 
ply the synthesis method outlined in Section 3 to check whether the two control 
problems defined above arc solvable. Denote the DTP corresponding to Aq by 
Mq and form the synchronous products Poa AIq || Sa and Pq6 Mq || Si, (the 
specification automata Sa and Sb are shown in Figure 5 and Figure 6, respective- 
ly). Xot surprisingly, the set of controllable and temporally nonblocking DSPs 
turns out to be empty for both Poa and Pob- on this (coarse) level of abstraction, 
neither of the two problems can be solved. 

Hence, we need a more accurate discrete approximation and compute the 
Moore automaton Ai. It consists of 67 states and 962 transitions. Denote the 
corresponding DTP by Mi and form Pia := Mi || Sa- Now, the set of all con- 
trollable and temporally nonblocking DSPs of P\a is non-empty. Its supremal 
element Pg realizes the least restrictive supervisory control scheme; it has 36 
states and 123 transitions. The supervisor action can be inferred from a table 
that lists possible control inputs for each of these states - an excerpt is shown 
in Table 2: if, for example, only one measurement symbol has been observed 
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Table 2. Possible control symbols (excerpt). 





supervisor state 
state of Al state of Sa 


possible control symbols 


yd{tk- 


i) yd{tk) Ud{tk-i) 






- 


vP 


^21 

Xs 


(9] 

-“d 


vP 

yP 


yP ^P 
vP uP 


^(3) 

Xs 

^(3) 

Xs 


(3) 

-“d 

(9) 

«d 


vP 


... . . 


^(4) 

Xs 


(3) 

«d 



{tk = to), if it indicates that the water level in tank 3 is “low” [yd{tk) ~ Vd'’)^ 
and if the specification automaton Sa is in state Xg ^ , the only possible control 
action is (“switch both pumps to full flow rate”). If, on the other hand, the 
observed plant state is {[y^p and Sa is in state x^p , the supervisor 

allows the control symbols and any one of them can be picked by an 
arbitrary selection mechanism (if there arc several options, we always choose the 
first line)- both of them are guaranteed not to violate the specification. 

Figure 7 illustrates how the control scheme works: it shows a closed loop 




Fig. 7. Simulation of closed loop {xi{t) is dashdotted, X 2 {t) dashed, xo{t) solid). 
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simulation of the continuous plant model (17) — (24) under discrete supervisory 
control. We start with an initial condition x(0) — [10cm 40cm 10cm]. Hence, 
the first available measurement information is yd(to = 0) = (i.e. water level 

in tank 3 is low). The first occurrence of a measurement symbol drives Sa into 
state The supervisor responds with control symbol The water level 
starts to rise in tank 1 and 3, and the next available measurement information 
(at time H = 100s) is The new plant state is [w|f^]), and Sa has 

switched to state Xg . Accordingly, the control input is changed to , and the 

(Q.\ 

plant again responds with yd{t 2 ) — Ud ^ 

We apply the same procedure for the second specification: the set of control- 
lable and temporally nonblocking DSPs of Fif, := M\ || Si, is also non-empty. 
Its supremal element has 86 states and 251 transitions. Figure 8 shows a sim- 
ulation of the continuous plant model (17) - (24) under discrete control; the 
initial conditions for the simulation are x(0) = [10cm 80cm 50cm]'. Again, as in 
the previous case, the controller consists of the supervisor Ps and an arbitrary 
mechanism that picks any one of the control symbols that Ps allows. Clearly, 
the closed loop satisfies the specifications. 




Fig. 8. Simulation of closed loop {xi{t) is dashdotted, X 2 {t) dashed, X 3 {t) solid). 



5 Safety Enforcement for an Evaporator 

5.1 Plant and Continuous Model 



The second example is an evaporator for a sodium chloride solution (Figure 9). 
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Fig. 9. Evaporator. 



It has been suggested in [7] as a benchmark problem for the synthesis of 
discrete event supervisors and sequential logic controllers in the process indus- 
tries. The process is meant to be run in batch mode: the evaporator is filled, 
salt concentration is increased to a desired level by boiling, then the evaporator 
is emptied, and the cycle starts again. We only consider the safety enforcement 
part for this problem. Safety enforcement prohibits any situation which is re- 
garded to be unsafe or undesirable and hence provides a (minimally restrictive) 
framework, within which a lower level (sequential logic) controller operates and 
forces the system into the appropriate batch cycle. 

As state variables, we choose molar holdup {ul [mol]) , molar concentration 
of sodium chloride in the liquid phase [xg [mol/kmol]), and temperature (T [K]). 
Hence, the state set X of the continuous model is a dense subset of K^. Our 
continuous model for the evaporator is based on balance equations: 

1. total material balance [mol/s]: 



driL 
d t 



= FL,in ® Fl ® Fv , 



(25) 



2. component material balance of sodium chloride [mol/(kmol s)]: 



d X 1 

T7~ — ^ ^.s) Fy^s) i 

dt ul 



(26) 
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3. energy balance [.7/s]: 
d T 

= FL,tn{hL,in^ ^l) 'S) FyAh 
d Ah , „ , \ \ 

“I ^ ^s) “i“ FyXs) 

+ Qheat ® a(A^ + )(y ^ Tamb) ■ (27) 

A" PL 

Liquid and vapour flow {Fl and Fy) are auxiliary variables, Xs,in, hL.in, 
a, Ay , D'^ , and Tamb constant parameters, Ml, Cp^L, hL, Ah and pL physical 
properties. All of them are explained in the Appendix. 

Heating power Qheat [^], position Y [®] of the drain valve, and molar liq- 
uid flow FL,in [mol/s] are control inputs. They are all operated in an “on/off” 
manner: 



Qheat e {0, 60001T} 

e {0, 1} 

FL,in e {0, 2mol/s). 

The latter variable corresponds to the position of the inlet valve Vi, its possible 
values can be realized by opening or closing Vi . If we exclude combinations where 
both valves are open (this doesn’t make physical sense), we have six different 
control symbols, denoted by (Table 3): 

Table 3. Control symbols for the evaporator problem. 



:= “valve Vi closed” & “valve V2 closed” & “heating off” 
■= “valve Vi closed” & “valve V2 closed” & “heating on” 
:= “valve V\ closed” & “valve V2 open” & “heating off” 

:= “valve Vi closed” & “valve V2 open” & “heating on” 

:= “valve Vi open” & “valve V2 closed” & “heating off” 

:= “valve Vi open” & “valve V2 closed” & “heating on” 



We sample the plant at intervals of 20s, i.e. the control signal is changed and 
measurement information is available only at the sampling instants to = 0,ti = 
20s, t 2 — 40s, .... Measurement information consists of quantized versions of 
liquid volume Vl = ^Ml and salt concentration Xg. This is illustrated in 
Figure 10. 

5.2 Specifications 

In this example, specifications can be easily expressed in terms of measurement 
and control symbols: 
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86.4 97.2 108.0 



Fig. 10. Measurement symbols. 



Spec. 1: the heating must not be switched “on” if ^0.95; this translates 

into Ud{tk) ^ {u^P,Ud\ud'’} if Vditk) e {yaKyd^Vd'’}- 

! 

Spec. 2: overflow has to be avoided, i.e. Vl ^8.55 or, equivalently, yd{tk) 0 

! 

Spec. 3: crystallization must not occur, i.e. Xg -^97.2 or, equivalently yd{tk) ^ 



5.3 Discrete Abstractions and Control 

As in the three-tank example, abstraction Aq is too coarse for the control prob- 
lem at hand. Hence, we compute Ai. It has 236 states and 2196 transitions. 
Recall that abstraction states arc simply collections of measurement and control 
symbols. Hence, with all the specifications being static, they can be conveniently 
translated into “forbidden state specifications”: Spec 1, for example, attaches the 
label “forbidden” to all states of the form Xd — y^d = 1j2,3, 

fc = 2,4, 6. Now, A\ is converted into the corresponding DTP Mi; the set of all 
safe (recall that “safe” means that all “forbidden” states have been removed), 
controllable, and temporally nonblocking DSPs of Mi is nonempty. Its supre- 
mal element Ms corresponds to the maximally permissive safety enforcement 
scheme. It has 75 states, and a small excerpt is shown in Table 4. If the plant 
(and hence the supervisor) is in state ^ y^P ^y^P > ^lP j , all possible control 
actions arc deemed to be safe and therefore allowed. If the observed plant state 
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is ^ ) 5 control symbols and are disabled, i.e. valve 

Vi is closed to prevent overflow. 

A vital requirement for any safety enforcement scheme is of course that it 
“leaves enough degrees of freedom” to an underlying (sequential logic) controller. 
In our example, this implies that the desired batch mode must still be possible 
under supervision. As Figure 11 illustrates, this is indeed the case: we can fill 
the tank to a desired level, increase salt concentration (mole fraction) by boiling, 
empty the tank, and restart the cycle within the safety enforcing framework of 
a discrete supervisor. 

6 Conclusions 

An approximation based approach to hybrid control systems design was success- 
fully applied to two examples from process control - safety enforcement for an 
evaporator and supervisory control of a three-tank laboratory experiment. The 
approach guarantees that any (discrete) feedback law that solves the control 
problem on the level of the discrete abstractions will also force the continuous 
plant model to obey the specifications. In both examples, the underlying contin- 
uous model was represented by detailed nonlinear equations. Especially in the 
evaporator case, the nonlinearities were “severe” . Nevertheless, our algorithm 
proved to be able to handle the approximation step in reasonable time. Control 
synthesis on the discrete (abstraction) level was performed using an adapta- 
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Fig. 11. Batch cycle for the evaporator system under discrete supervision 



tion of the Ramadge/Wonharn framework. Our synthesis software proved to be 
extremely efficient, and computation time was negligible. 
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A Details of the Evaporator Model 

In this section, we specify all the details for the evaporator model. We list e- 

quations for auxiliary variables and physical property correlations (Sections A.l 

and A. 2), and collect all parameters in Section A. 3. 



A.l Equations for Auxiliary Variables 

Vapour and liquid flow {Fy [^^], Fp [^^]) are determined by 

^ 1 4.10^ RT jp^Pamb) 

Vy y {P Pamb^ 

fl = ( + + )/(2ei), 

where pressure p [^j^] is given by 

p = rnax(pamb, Ps,l)- 



(28) 

(29) 



(30) 



Vy (molar volume of the vapour phase) and ps.L (saturation pressure of sodium 
chloride solution) are additional physical properties (see section A. 2). 
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A. 2 Physical Property Correlations 

The following physical property correlations have been used (throughout this 
section, xs is in ^): 

— Molecular weight of sodium chloride solution 

Ml = Xs {Ms ® My,) + My, . (31) 

— The density of sodium chloride solution {pl [^^]) is determined according 
to [10]: 



6 2 

PL = Pw{Tr) + (T® 273.15)* + . (32) 

i=i j=i 

The constants in (32) were fitted using data from [18] and [20]: 

Pw{Tr) = 999.858, kp^, = 0.600919 □ lO-^, 

^P „2 = ®0. 919335 D 10-2, ^ ^ 0.133038 □ 10 -^ 

= ®0.21617D 10-5, = 0.197676 □ 10-^ 

kp^.6 = ®0. 70207 D 10-10, kp^', = ®2241.66, 
fcp, 2 = 2190.21 . 

— Specific heat capacity of sodium chloride solution Cp,L[-^] [4]: 

2 2 

= (33) 

i — 0 j —0 

fccp , 00 = 0.136538 D lOi, /ccp,oi = ® 0. 8959483 D 10 ^, 

kcy , 02 = 0.2380725 □ lO^, k^Jpo = ® 0. 2032837 D IQ-^, 

= 0.3627181 □ 10-1, /ccji2 = ®0. 6216819 D 
kcj,20 = 0.32 2 1 8 32 □ 10-5, ^ 00.6152962 D 10 -*i, 

fccp!22 = 0.1055741 □ 10-3. 

— Molar enthalpy of vapourization of sodium chloride solution Ah [:^^] [5]: 

^ jlf ^ 

Ah = MLY.kAhA^T^y (34) 

i =0 ^ 

kAhfio = 0.2256215 lO^, kAh.io = 00.1860691 lO^, 
k Ah, 20 = 0.5003167101, kAh.oo = 00.2713489 105, 
fczi+40 = 0.5114429105. 

— Molar enthalpy of vapour phase of water hy [ 7 ;^]: 

3 

hv^'^khy,i (T 0 273.15)*. 

1=0 



(35) 
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khvfi = 45757.0, khv.i = 32.89359, 

= 5.768308 □ 10-3, = 07,886424 D ^q-®. 



— Molar enthalpy of liquid phase of sodium chloride solution [ 7 ^]: 



= hv ® . 



(36) 



— Saturation pressure of sodium chloride solution [d^] I^]' 



-I p.3 Xs~tfcs,g,2 ) (ks ,w ,1 T + k ' y , s s ,4 Xg “tfcs.s.5 Xg) 

Ps,L = iu e 

(37) 

ks,w,i = 0.23478 D 10^, = 0.39849 D 10^ 

fcsiwia = ®0. 39724 D 10^, = ®0. 33204656 D 10-\ 

2 = 0.13635246, = «0. 19203616 D lO^ 

= 0.32750375 D 10\ fc.’s’s = ®0. 19866438 D 10^. 

— Molar volume of ideal gas vy [dm^ /mol]: 



vv 



RTIO 

P 



(38) 



A. 3 Parameters 

To completely specify the evaporator model, we need a number of parameters. 
They are listed in Table 5. 



Table 5. Parameters of evaporator model. 



A* = 0.022698 dm'^ 
A" = 3.14159 
= 2 dm 
g = 98.1 ^ 

?o = 0.8892 gj* 



cross sectional area of vapour tube 

cross sectional area of evaporator 

diameter of evaporator 

gravitational constant 

pressure drop coefficient 

pressure drop coefficient 

molecular weight of water 

molecular weight of sodium chloride 

gas constant 

heat transfer coefficient 

ambient temperature 

ambient pressure 

molar enthalpy of inlet flow 

mole fraction of sodium chloride in inlet 



flow 



mqj 

3.23712 ^ 

dm^ moL-^ 



Cl 

= 18.01488 
Ms = 58.4428 
R = 8.3145 
a = 0.02 



rnol 
jnol 



dm^K 



ol K 



Tamb = 293.15 K 
Pamb = 1013.0 
hL.in = 7243.543 ■ 
Xs = 0.925545 



kmol 
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Abstract. An action system framework is a predicate transformer based 
method for modelling and analysing distributed and reactive systems. 
The actions are statements in Dijkstra’s guarded coimnarid language, and 
their semantics is given by predicate transformers. We extend conven- 
tional action systems with a differential action consisting of a differential 
equation and an evolution guard. The semantics is given by a weakest 
liberal precondition transformer, because it is not always desirable that 
differential actions terminate. It is shown that the proposed differential 
action has a semantics which corresponds to a discrete approximation 
when the discrete step size goes to zero. 

The extension gives action systems the power to model real-time clocks 
and continuous evolutions within hybrid systems. In this paper we give 
a standard form for such a hybrid action system. We also extend parallel 
composition to hybrid action systems. This does not change the origi- 
nal meaning of the parallel composition, and therefore ordinary action 
systems compose in parallel with hybrid action systems. 



1 Introduction 

Action systems, originally proposed by Back and Kurki-Suonio [3], are predicate 
transformer based systems for modelling discrete computations. For a given pro- 
gram, the predicate has the program variables as free variables. The transformer 
uses Dijkstra’s idea of describing the effect of a given program by calculating the 
precondition which guarantees that a given postcondition holds. Action systems 
have been extensively used in sequential program rehnement, and also in reactive 
and concurrent programs, see e.g. [4,9]. 

Previously action systems have been extended with explicit clock variables 
when reasoning about real-time systems, and with discrete approximations when 
modelling continuous phenomena in hybrid settings [13,19]. However, since there 
is no technical reason why predicates over program variables should not speak of 
continuous entities, it seems possible to simplify reasoning in these applications 
by introducing the differential equations governing evolution of clocks and other 
continuous variables directly into the framework. Such an extension is the main 
result of this paper. A guideline in developing the differential action is a wish to 
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preserve the connection between discrete accumulation and continuous evolution, 
such that the weakest liberal precondition semantics corresponds to the one 
found for the approximating discrete accumulation. 

Inspired by the Branicky's unified model for hybrid systems [6], we identify 
a class of action systems suitable for modelling hybrid systems. These hybrid 
action systems alternate between discrete changes and continuous evolutions. 

The parallel composition for ordinary action systems is described by inter- 
leaving of actions from the constituent systems. This model is based on the 
assumption that the effect of two simultaneous actions is the same as the effect 
of the two actions in some sequence. This assumption is not valid in a continuous 
parallel world, where the total effect of simultaneous differential actions is the 
sum of their individual effects. This form of composition is captured by a lin- 
ear composition. We extend the parallel composition for hybrid action systems 
so, that it uses linear composition for combining the continuous evolutions, but 
interleaves discrete actions. We retain the usual meaning of the parallel composi- 
tion for ordinary action systems. Hence, the mixed parallel composition between 
action systems and hybrid action systems remain also meaningful. 



Related work. Action systems are similar to the UNITY programs [10]. Espe- 
cially, the rehnement methodology is central in both of these approaches. In [11] 
is a discussion of how a program logic may be used to reason about continuous 
systems. That paper focuses on exploring the potentials of refinement reasoning 
for such systems, whereas we concentrate on a precise definition of the semantics. 

Action systems have an operational flavour, and our extension is thus related 
to various hybrid automata models [1,20,18] and the general hybrid model sur- 
veyed in [7]. However, our interest is not in the basic model, but in an integration 
of specifications (predicates) and actions such that one can reason without ref- 
erence to the semantics. A similar concept is found in the description language 
used in HyTech [15,16], where the approximations formulated in [17] would be 
seen as action refinements. In applications, it seems promising to first use ac- 
tion refinement and standard mathematical analysis tools to weaken complex 
predicates with differential equations and then translate to HyTech and use its 
potential for automated checking. 

A directly related piece of work is the duration calculus semantics for hybrid 
eSP [23,14], which, however, does not include a proof theory. The current paper 
may be seen as an investigation of a proof theory analogous to the one investi- 
gated for duration calculus in [22]. However, that work does not include parallel 
composition. 



Overview. We start by defining action systems in Section 2. In Section 3 we de- 
rive from the weakest liberal precondition for a discrete accumulation a predicate 
describing continuous evolution. Based on this predicate we define the differen- 
tial action. In Section 4 we study a gate example [21] to illustrate the similarities 
and the differences between formulations using discrete accumulation and con- 
tinuous evolution. In Section 5 we identify a suitable form to describe hybrid 
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systems. We call such action systems hybrid action systems. In Section 6 we 
investigate the parallel composition of action systems and extend it so that it 
is meaningful also when composing hybrid action systems. In Section 7 we use 
a water tank example to illustrate the extended parallel composition. We also 
use the example to outline, how HyTech can be used in analysing hybrid action 
systems. Section 8 concludes and discusses directions for further work. 



2 Actions 

An action is any statement in Dijkstra’s guarded command language [12], in- 
cluding pure guarded commands. In a system, the actions operate on a fixed 
set of variables. The values of these variables identify a state in the system. 
Therefore, predicates or conditions on these variables provide a convenient way 
of specifying and reasoning about states in a system. 

When we consider a given action A, we may speak of a postcondition q. It is 
a predicate specifying desirable states after the execution of the action. By using 
the postcondition as a reference, we can define the meaning or the semantics for 
the action A. We can, for example, use the weakest precondition predicate trans- 
former wp [A, q) for this purpose. It returns a predicate describing the largest 
set of states from which the action terminates and reaches the postcondition. 

The weakest precondition can also be used for calculating two basic properties 
for any action. These properties are the termination set and the enabledness set. 
The set of states from which an action terminates, i.e. reaches an arbitrary state, 
is given by tA = wp [A, true) . Similarly, the set of states in which an action 
is enabled, i.e. the states from which an action starts to execute, is given by 
gA = ^wp(A, false). 

In the following we give the semantics for the actions with the weakest liberal 
precondition, wlp. Unlike the weakest precondition, it does not require termina- 
tion. The relation between these two predicate transformers is given by Dijkstra 
[12] as wp {A, q) = wlp (A, q) A tA or equivalently when A is conjunctive 
wlp (A, q) = (t A ^ wp(A, q)). 

Furthermore, we consider two actions A and B to be the same when their 
semantics agree for any postcondition in the sense of weakest liberal precondition: 

A = B iff V g ■ wlp (A, q) =wlp(i3, q) 

Any two actions for which this relation holds can be substituted for each other 
in any context without affecting the meaning. 



2.1 Semantics for actions 

In this paper we use capital letters such as X, U, E, and F to denote a vector 
or list of variables and other components. For clarity, we do not use subscripts 
when referring to the components in a vector; we explicitly say which small 
letters denote vector components, whenever this is needed. Furthermore, let 
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q[E / X] denote textual substitution of expressions E for the free variables X in 
a predicate q. 

Elementary actions arc defined as in [5,8]. 

wlp (abort, q) = true 

wlp (skip, q) = q 

wlp (A := E, q) C q[E/X] 

Note that abort never terminates, whereas a skip or an assignment are sure to 
terminate. 

Let, as before, A and B be actions, and p be a predicate. The sequential 
composition, the composition by a non-deterministic choice, as well as prefixing 
an action with a guard are defined by 

wlp (A; B, q) = wlp {A, wlp {B, q) ) 
wlp (A I B, q) = wlp (A, q) A wlp {B. q) 
wlp (p -+ A, g) = p ^ wlp (A, g) 

An assertion ensures that a given condition p holds in the current state. If 
the given condition does not hold, the assertion aborts. Hence, the meaning for 
the assertion is {p} -i p ^ skip [ ^p ^ abort From which we calculate the 
semantics: 

wlp ({p}, g) = wlp (p —> skip, g) A wlp (^p ^ abort, g) 

= (p => wlp (skip, g) ) A true 

resulting in the semantics wlp ({p}, q) = (p ^ q)- 
Iteration has the liberal semantics 

wlp (do A od, g) = Vn>0 • wlp (A", gA V g) 

where A® = skip and = A; A". The semantics for iteration is useful 

when proving some general properties related to the iteration of actions. For 
example, we can easily show that if p is an invariant for the iterated action, 
p wlp (A, p) , it is also an invariant throughout the entire iteration, 
p => wlp (do A od, p) . An iteration terminates when it is no longer enabled, 
t(do A od) = 3 n -1 0 -i ^ gA". 



2.2 Action systems 

An action system A is an initialised block of the form 

A= |[ var A : T 
X ■- E- 

do Ai I . . . I A„od 
]\-.Z 

The expression var X : T , declares the list of variables X with types T. Any of 
these variables can be global. This is denoted by a star (e.g. x*} in the declaration. 
A global variable can be shared with other action systems. The global variables 
that do not belong to A, but are used by the actions Ai ... A„, are in the 
imported global variables Z. The variables in X that are not global, arc local for 
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the block. The local variables do not exist outside the action system where they 
are defined. All the local and global variables form the state space of an action 
system. 

The action X := E initialises the variables X by expressions E. Therefore, 
the global variables are initialised in the action system, where they are declared. 
Because the initialisation is a multiple assignment, the expression E may not 
refer to any variable in X or Z . For example, an initialisation x,y := x + 3, y + 4 , 
where {x, y} G X, is illegal, since the values of the variables are undefined before 
the initialisation. All action systems initialise before any action is taken in one 
of them. This becomes apparent later, when the parallel composition for the 
action systems is defined. After initialisation, the actions Ai ... An are repeatedly 
executed when enabled. The selection of an enabled action for execution is non- 
deterministic. There are no fairness assumptions about selection of actions. The 
execution of an elementary action is always atomic. This ensures that during the 
execution, the values of the variables remain unchanged, unless the action itself 
changes them. If two enabled actions refer to disjoint variables, their execution 
can be in any order or in parallel. Hence, this models parallelism by interleaving. 

The execution of the action system terminates when none of the actions is 
enabled, i.e. ^g(Ai [ ... [ A„). Similarly, the execution of the action system 
aborts when an executed action aborts. 

3 Differential Action 

We extend the conventional set of actions with a differential action that describes 
continuous behaviour. In the following we derive the semantics for the differential 
action from a discrete accumulation that approximates the continuous behaviour. 
We also show some of the useful properties related to the use of differential 
actions. 

3.1 Discrete accumulation 

In discrete accumulation a variable x is incremented from an initial value un- 
til a guard ceases to hold (if ever). When a discrete accumulation is used to 
approximate a continuous behaviour, the increment f{x) is approximately the 
differential multiplied by a fixed, positive time step h. Both the increment f{x) 
and the guard e may depend on h. The discrete accumulation is captured by 

do e — > X := X + f(x) ■ h od 

The meaning of the discrete accumulation is given by the weakest liberal 
precondition for the iteration action. Let p[v/xY denote the i times repeated 
substitution p[v / x]...\v / x], and n and m be natural numbers. By unfolding the 
precondition for the action above we get 

wlp (do e — > X ■.= X + f{x) ■ h od, q) 

= Vn>0 ■ (e V q)[x + hf{x)/x]^ V 30<m<n ■ -ie[x + ft/(x)/x]'" 



( 1 ) 
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Analysis of this predicate is laborious, because it requires the calculation of a 
predicate after repeated substitutions. Fortunately, we can express the repeated 
substitution as a single substitution with the help of a recursive function, 
which for the fixed h > 0 is given by: 



<Ph{n-h) = x + h)) 

This function has the property that <Ph{n^i) equals x[x+hf{x)/x]'^. The function 
also has an implicit form: 

$h{i ■ h + h) - ■ h) 



#h(0) = X, Vi > 0 



= f[^h{i ■ h)) 



(2) 



It is this form that we use in the concise semantics for the discrete accumulation. 
From the precondition expression (1) above we obtain 

wlp (do e — > X := X + /(x) ■ h od, q) (3) 

= 3$h ■ <Ph{0) = x A Vi>0 • <fhV-h+h)^-<PhV-h) ^ 

(Vn>0 ■ (e V q)[<Ph{n ■ h)/x] V 30<m<n • ^e[^h(m ■ ft)/x]) 



3.2 Evolution 

In a continuous evolution, the variable x is assumed to be a differentiable function 
of time. If a solution exists to the differential equation x = f{x), then it is the 
limit of the step functions \ h > 0) a.s h approaches zero. We denote this 
limit by and by applying the definition of differentiation to the formula (2), 
we have 

^>(0) = X, <P = f{<P) 

This observation leads us to define a new action for continuously evolving 
variables, called a differential action. The differential action evolves continuously 
the values of given variables X as long as a guard e holds, this guard condition 
may use the variables X. The evolution is controlled by a system of differential 
equations. We use the standard convention of writing a system of differential 
equations as X = F{X). The AT is a component-wise first derivative of X. 

The exploration above is summarised in the following definition. 

Definition 1. Let e he a guard and q he a postcondition, hath may speak of the 
variables X. The differential action e : X — F{X) has the semantics 

wlp [e : X = F{X), q) = 

3<L • <Pff) = X A d> = F{<P) A 
(Vt > 0 ■ (e V g)[^(r)/X] V 30<(5<r ■ -^e[L>{5) / X]) 

In the semantics, the first two conjuncts require the existence of the solution 
for the differential equation X — F{X). The last term isolates the first point 
in time, where the guard ceases to hold. 

It is difficult to capture the exact link between discrete accumulation and 
the differential action in the general case; and one cannot expect it to be easy, 
because it would answer many questions of numerical analysis. However, as an 
indication of the kind of result one may have, we give the following theorem. 
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Theorem 1. If the discrete accum,ulatioii, function I? h, given by 
<Ph{nh) = x + 

converges to a solution <1 for the differential equation x = f{x) from above, i.e., 
WhW ^ 1^1 for all h, and if the evolution guard e is upward closed, i.e, ||a|| ^ ||6|| 
implies e\ajx\ ^ e[&/x]; that is if e is true for some value, then it is true for any 
larger value. 

Then differential termination: ^wlp{e : x = f{x), false) implies accumula- 
tion termination: ~^wlp{do e ^ x := x + f(x) —h, od, false). 

The implication holds in the reverse direction if ‘from below’ and ‘downward 
closed’ are substituted for ‘from above’ and ‘upward closed’. 

3.3 Stability 

If the evolution guard e remains true in a differential action, it never terminates, 
as seen from wlp (e : X = F{X), false) — true-, but the behaviour is bounded 
by the guard. In this case, the system becomes stable with respect to the guard. 
When something is known about the stability of the differential equation, it may 
be used in the analysis of the differential action. 

Theorem 2. Assume that the differential equation X — F{X) has solutions. 
Let O be a point in the domain of X . Let a sphere around O with radius e, be 
denoted by S{e). 

Assuming that a solution with initial value Xq in S{p) is Ljapunov stable in 
S{e), then we have 

wlp ( {X e S{p)}: X G S'(e) : X — F{X), false ) — true 
Furthermore, if the solution is asymptotically stable, then for any 7 > 0; 
wlp ( {X e 5(p)}; X 6 S{e) \ S{y) : X = F(X), X G 5'(7) ) = true 

Also, if the solution for X — F{X) is periodic and O is any point on the 
solution curve, 

wlp ( {X / O}; X ^ O : X = F(X), X = 0) = true 



3.4 Clocks 

A clock or timer that measures the passage of time is specified with a differential 
action. An accurately running timer c is given by e : c = 1. If it had a skew of 
at most 1 %, it would be {0.99 < 6 < 1.01}; e : c = 1-6. The assertion in front 
of the differential action ensures that the skew is within the given limits. 

A delay for four time units is given byc:=l; c<5: c=l. The corre- 
sponding discrete accumulation would have / = 1: 
do c < 5 ^ c := c + 1 ■ ft od 

and have the aeeumulation function: <L(n -h) = c + h ^ which reduces to 

<L{n -h) = c+h -n; as one would expect. It will terminate when n = [^^], i-e., 
not later than the corresponding differential action, as predicted by the second 
part of Theorem 1. 
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3.5 Composite evolutions 

The following is an example of an evolution of several continuous variables. There 
are two variables x and v that evolve. The x variable is dependent on the v. 
wlp {e \ x,v = v,l , q) 

= ■ S>x(0) = x A S>„(0) = n A = #„ A = 1 A 

(Vt> 0 ■ (e V q)[<Px{T),<l>v{T)/x,v] V 30<(5<t ■ ^e[<^x{6),'^v{5) /x,v\) 

If we assume that the action has been preceded by an initialisation, say 
X, V := 0, 0, the clock example allows us to simplify the result to 
■ ^x(pi) = Q A i>x = v A 

(Vt > 0 ■ (e V q)[(l>x{T),T /x,v\ V 30<(5<t • ^e[(l>x(5),S /x,v\) 

In an application, the timer v is probably there to limit the evolution of the 
time varying system x. This could result in an guard e = (w < 5), for which we 
would get: 

3^?* • <p3:(0) = 0 A <i>x = V A q[<Px{^),^/x,v] 

Notice that 5/x, u] specifies all those states, from which x evolves to 

values •I’xi'o), when v evolves linearly to value 5. 

3.6 Algebraic Properties 

In the following we assume that the systems of differential equations used in 
the differential actions have solutions. Also, the equivalences are in the sense of 
weakest liberal precondition as defined in Section 2. 

A differential action acts as a skip action when the evolution guard does not 
hold initially. This is captured by the following equivalence, which can eliminate 
a redundant differential action. 

{-le}; e ■. X = F{x) = {le}; skip (4) 

At termination of a differential action the evolution guard does not hold any- 
more. This is captured by the following equivalence, which is used for introducing 
a termination assertion. 

e : X = F{x) = e : X = F{x); {^e} (5) 

With the help of the two properties above, it is easy to prove that sequen- 
tial composition of a differential action with itself has the same effect as the 
differential action by itself. 

e : X = F{x); e : X = F{x) 

= { termination assertion introduction (5) } 
e : X = F{x); {^e}; e : X = F{x) 

= { redundancy elimination (4) } 
e : X = F{x)] skip 

= { skip elimination } 
e : X = F{x)] {-'c} 

= { termination assertion removal (5) } 
e: X = F{x) 




312 



M. Ronkko and A.P. Ravn 



This illustrates that sequential composition takes no time; there is no evolution 
over the composition. 

A differential action docs not change the values of variables that do not 
belong to X. Therefore, we can we can weaken or strengthen the evolution guard 
by moving parts to or from an assertion. Let 5 be a predicate not containing 
variables of X, then the following holds for a differential action 

{g}; g A e: X = F{x) = {g}; e: X = F{x) 



4 A Gate Example 

The following example illustrates the differences and similarities between a dis- 
crete and a continuous approach in modelling and analysing a system. We use 
the lowering of a gate as an example. The gate is lowered with a speed propor- 
tional to its current position, that is i = The task is to model the movement, 
and measure the time for the gate to become closed. The gate is started 1 meter 
above the stop and it is considered to be closed, when it is at most 1 cm away 
from the stop. Figure 1 illustrates the setting. 




Fig. 1. The gate. 



4.1 With discrete accumulation 

Let, as before, h be the time step. With discrete accumulation the model is 
X, c := 1, 0; do a; > 0.01 ^x,c:=x — x-h,c+l-h od 

The time consumed is given by c, which is n -h, where n is the number of 
iterations; but we cannot use this directly, because n depends on h. 

Wc therefore analyse the scries of substitutions taking place in the weakest 
liberal precondition for the discrete accumulation: 

X, X — x ■ h, X — X ■ h — {x — X ■ h) ■ h, 

{x — X ■ h — {x — x ■ h) ■ h) — (x — X ■ h — (x — X ■ h) ■ h) ■ h, 

After taking common factors and performing series simplification, we obtain 
an accumulation function ^(n —h) = x{\ ® /i)". Next wc solve the h from the 
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accumulation guard 1 -{1 ® /i)" = 0.01 yielding h — 1® ino.oi Replacing the 
h in expression for c and simplifying it gives c = n(l ® e~ This expresses 

the time consumed by the lowering of the gate in proportion to the iteration 
accuracy n. Taking the limit of the Taylor polynomial of this expression when n 
approaches infinity gives an approximation for the total time, (8> In 0.01 4.61. 

4.2 With continuous evolution 

The same problem is expressed with continuous evolution by 
X, c := 1, 0; X > 0.01 : x,c = —x, 1 

The total time is analysed as follows: 

wlp ( X, c := 1, 0; x > 0.01 : x, c = — x, 1 , x < 0.01 A r = c ) 

= I definitions, instantiation of <Px{t) = xe^‘ and 'Pc{t) = c + t } 

Vr > 0 ■ > 0.01 V r = r V 30<5<t ■ < 0.01 

= { case analysis around r = — In 0.01 } 
r = — In 0.01 

This tells us directly that after ® In 0.01 time units the gate becomes closed. 

5 Hybrid Action System 

A hybrid system starts from some initial state and alternates between continu- 
ous evolution and discrete changes of the variables in the system. The various 
circumstances, under which this alternation takes place, are studied more closely 
by Branicky [6]. 

5.1 Stutter free evolution 

A differential action describes how the variables evolve when the guard is en- 
abled. It also states that when the guard does not hold, the differential action 
has no effect on the variables. This is called stuttering. The equivalence below 
shows the decomposition of a differential action. 

e-. X = F{X) = e^e: X = F(X) [ ^ skip 

When thinking of how a hybrid system behaves, we are mainly interested in 
the stutter free behaviour. An infinite stuttering is seen as termination. There- 
fore, we introduce a notational short form for a stutter free differential action, 
which is e A = F{X) with the meaning of e ^ e : X = F{X). The stutter 
free differential action has the property that it is enabled only when the evo- 
lution guard holds, that is g(e X — F{X)) — e. Also, the enabledness 

set of a non-deterministic composition of stutter free differential actions is the 
disjunction of the evolution guards, that is, 

g(ei X = Fi(X) 1 ... 1 e„ X = F„(X)) = 



ei V ... V e„ 
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5.2 Hybrid alternation 

The alternation between discrete and continuous behaviour is expressed by it- 
eration of ordinary and (stutter free) differential actions. Assuming that the 
cause for the discrete behaviour D is calculated to be gD, and the stutter free 
evolution is known as DA, the alternation takes a normal form 

do gD D (6) 

1 ^gD^DA 

od 

The D in the model may be a non-deterministic composition of several discrete 
actions. Similarly, the stutter free evolution DA may be a non-deterministic 
composition of several stutter free differential actions. 

For the form (6) of alternation above we introduce a short form 
alt D with DA 

This construct gives discrete changes priority over the continuous evolution. 
The continuous evolution may proceed only after all enabled discrete changes 
have been processed. However, once the continuous evolution is engaged it may 
continue without any interruption till the selected differential action terminates. 
After termination, the discrete changes may occur again. Thus, the model inserts 
discrete computations in between any continuous evolution described by a dif- 
ferential action. The alternation terminates, when it reaches a state in whieh no 
discrete changes and no continuous evolution is enabled, that is -i gD A gDA. 
Hence, all the initial states from which the system terminates are given by 
t(alt D with DA). 

The time in the model is assumed to progress during the continuous evolution, 
but not during the discrete changes. Therefore, if the discrete changes do not 
disable themselves, wp [D, -^gD) = false, there is no continuous evolution nor 
a termination after the discrete changes become enabled. This corresponds to 
the system being aborted. 

With the given form of alternation (6) we can identify a special class of action 
systems which have the desired hybrid behaviour. 

Definition 2. Let X be local variables and Z be imported global variables. The 
actions in D and DA may refer to all of these variables X or Z . Any action 
system of the following form is called a hybrid action system. 

H A |[varX:T 
X := 7; 

alt D with DA 

]\:Z 

When there are no discrete actions, i.e., gD — false, the alt part is dropped. 
Similarly, the with part is dropped, when there are no continuous evolutions, i.e., 
gDA = false. 

A hybrid action system may speak also of the continuous evolution of im- 
ported global variables. The meaning of a hybrid action system is obtained by 
unfolding the hybrid alternation into an ordinary iteration (6). This is important 
for the dchnition of a non-trivial parallel composition. 
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6 Parallel Composition 

Parallel composition is defined between action systems in [2]. Consider two action 
systems A and B 

A=\[varX:T B = \[ var Y : S 

X := r, Y := J; 

do Ai 1 . . . 1 Am od do Bi [ . . . 1 od 

]|:Z ]|:f/ 

where Z are imported global variables to A and U arc imported global variables 
to B, and the local variables X and Y are assumed to be disjoint. The parallel 
composition ^ || S is defined to be the action system 

A\\B = |[ var X,Y :T,S 

X, Y ■- I, J; 

do Ai I . . . I Am 1 Bi 1 . . . I od 
]\:{ZUU)~ {X\JY) 

The parallel composition combines the state spaces of the two action system- 
s, merging the global variables and keeping the local variables distinct. Those 
imported global variables in Z U C/, that are defined in either of the actions 
system, i.e. {X U Y), are now locally declared. 

The behaviour of the parallel composition depends on how the individual 
action systems, the reactive components, interact with each other via the global 
variables that they use. For instance, a reactive component does not terminate 
by itself. Termination is a global property of the composed action system [2] . 

Since the initialisations are well defined, and non-deterministic choice be- 
tween actions is both associative and commutative, the parallel composition is 
also associative and commutative. Therefore, the meaning of several parallel 
composed action systems can be unfolded in any order without affecting the 
result. 



6.1 Hybrid Action Systems 

The parallel composition for action systems uses interleaving for parallelism. 
This works well for a discrete behaviour, but not for a continuous behaviour. 
Consider the previously presented gate example. Suppose, we try to compose 
another force lifting the gate with a constant velocity 0.5 '"/s whenever the gate 
is at least half a meter away from being closed. Composing this force by inter- 
leaving the earlier non-linear closing force does not produce the desired result. 
By interleaving, the gate would non-deterministically either close normally, or it 
would start to open. In reality, we would expect that the gate stops half closed. 
The problem with the interleaving composition is that it does not really allow 
a simultaneous effect of the components, which is what we would like to have 
with the continuous behaviour. Motivated by this observation we explore an 
alternative way of composing continuous behaviour. 
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Linear Composition. In linear composition two partially defined functions are 
added on their common domain, whereas in the remaining domains the functions 
retain their original meaning. Linear composition thus assumes that a function is 
zero outside the domain where it is defined. The linear composition is formalised 
as follows. Let / be a partially defined function in the domain dom /. Also, let 
(/ be a partially defined function in the domain dom g. The linear composition 
of these functions is a set of partial functions 

{ /(x) + g{x), X € dom / n dom g 

/(x) + 0, X 6 dom / n dom g (7) 

0 + g(.x), .X e dom / n dom g 

The linear composition is both associative and commutative. It is defined in 
the whole range of dom / U dom g, whereas ordinary addition is defined only 
in the range dom / n dom g. 

Obviously, the linear composition can be generalised for partially defined 
systems of differential equations. Let T{X) and G{X) be sets of vectors 

! Fi{X), X E dom Fi ( G\{X), X E dom Gi 

G{X) = J : 

Fm{X),X Edom Fm [ G„(A), A G dom G„ 

where the domains dom F\ . . . dom are disjoint, as well as the domains 
dom Gi . . . dom G„ . A linearly composed system of differential equations 

A = A(A) e g{x) 



unfolds to a set of partially defined systems of differential equations 



A = A(A) + g{X), X E IJdom Fi n [Jdorn G, 

i=i j=i 



X = F{X), 
X = g(A), 



A G IJdom Fi n P|dom Gj 

i=i j=i 



X E fjdom Fi n IJdom Gj 

i=L j = l 



Linear composition applies to stutter free differential actions. A stutter free 
differential action e ^ X = F{x) denotes a partially defined system of differ- 
ential equations in the domain where e holds, that is {t \ e[J(f)/A]}. A set of 
such systems with disjoint domains is a set of differential actions combined with 
non-deterministic choice. 

Let DF be a composition fi X — Fi{X) [ ... [ X — F^(A), and 

DG be a composition gi X = Gi(A) j ... [ g„ X = G^{X). The DF 
and DG are analogous to X — F{X) and X — G(X). The linear composition of 
these functions presented above, is expressed in terms of actions as 
1 i = = l..n ■ fi A gj A = G(A) + Gj{X) 

1 i = l..rn ■ fi A -^gDG A = FJA) 

lj = l..n- -.gDF A gr.-^X^Gj{X) 

where gDF is fi V ... V fm and gDG is gi V ... V g^- 
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Example Let X be a vector of variables x, y, and 2 ;. Let F{X) be a vector of 
functions [f{X), g{X), 0), and G(X) be a vector of functions (0, h{X), u{X)). 
Consider the following linear composed system of differential equations where e 
and i arc predicates over X used for describing the domains. 

X = F{X), {X I e} © G(X), {X | i} 

This unfolds to 



X = 


if{X),g{X) + h{X), u{X)) 


, {X 1 e A i} 


X = 


(/(X),.g(X),0), 


{X 1 e A -if} 


X = 


(0, h{X), u(X)), 


< 

r 



We use the differential actions to denote the same linear composition of systems 
of differential equations as follows. 

e X = F{x) © i X = G(X) 

This unfolds into the following non-deterministic composition 

e A i :^x,y,z = J{X), g{X) + h{X),u{X) 

1 e A x,y,z = f{X),g{X),0 

I A i x,y,z = 0,h{X),u{X) 

6.2 Parallel composition of hybrid action systems 

The separation of the continuous evolution and the discrete changes in a hybrid 
action system allows an extension of the parallel composition. We extend the 
parallel operator so, that it composes linearly the continuous evolution, but 
interleaves the discrete changes. 

Let X and Y be disjoint local variables X, and Z and U be imported global 
variables. The ordinary actions H and stutter free differential actions DH may 
speak of variables X and Z, and similarly the ordinary actions K and stutter 
free differential actions DK may speak of variables Y and U . Consider the two 
hybrid action systems Ft and /C 

TfG |[varX:T XG |[varT:5' 

X := /; Y ■= J- 

alt H with DF[ alt K with DK 

1|:Z ]\:U 

where X and Y are disjoint. The parallel composition of these hybrid action 
systems has the following meaning. 

H\\IC = |[ var X,Y : T,S 

X,Y := 7, J; 

alt 77 1 X with DH © DK 

]\ : ZUU - {X UY) 

The meaning is obtained by first unfolding the linear composition for the con- 
tinuous evolution and then by unfolding the hybrid alternation as before. 
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Since both interleaving composition and linear composition are associative 
and commutative, this extended parallel composition is also associative and com- 
mutative. This means that the composition of several hybrid action systems can 
be unfolded in any order without affecting the result. 

A very simple example of parallel composition occurs, when we take an au- 
tonomous system, given by: 

S = |[ var X* : T 
X := I; 

with true X = F{X) 

]h 

and compose it with a control: 

C = |[ var R : S' 

R := J; 

alt gK K 

with e > X = (7 {X, R) 

]| :X 

The parallel composition gives: 

S II C = |[ var X*,V : T,S 

X,V ~ I,J; 
alt gK ^ K 

with true X = F{X) © e X = U[X, V) 

]l : 

which during evolutions satisfies X = F{X) + U (X, V). Control has been added. 

7 A Water Tank Example 

Parallel composition is illustrated by a water tank example. The water tank 
is a leaking container that is being filled, see Figure 2. Both leak and filling 
are continuous processes within the physical limitations of the tank. In this 
example, the task is to model these two different continuous processes as separate 
components and to obtain a model for the composed behaviour. 

At the beginning, the container is empty. It starts leaking as soon as the water 
level denoted by y is at or above the lower point of an outlet. This threshold 
is denoted by nmi. Filling starts right from the beginning, and stops when the 
water level reaches the lower point of the inlet. This water level is denoted by 
max. The water flows in at rate 1 */g and it flows out at rate ^ */«• 

7.1 Leaking 

The water level y is a global variable, so that the filling process can share it. 
The hybrid system for the leaking consists only of an evolution describing the 
water flow from the tank whenever the water level is at or above min. 

£eak = \ [ var y* : M 

y ■= 0; 

with min < y V ~ 
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Fig. 2. The water tank with threshold water levels {min and m,ax) and flow 
rates. 



7.2 Filling 

We use the global variable y from the previous action system to describe the 
water level. A global variable fill records whether the tank is being filled or not. 

The hybrid filling system alternates between a discrete decision to stop the 
filling when the water level hits the max threshold and a continuous evolution 
describing the water flow into the tank during the filling. 

IFill = \l~var fill* : IB 
fill := true; 

alt fill A y > max fill := false 
with fill A y < max :—> y = I 

]| : y 

The difference between this and a continuous evolution with y < max > y = 1 
is that the hybrid action system above fills the tank only once. 

7.3 Tank 

The behaviour for the water tank is obtained through a parallel composition of 
the models for filling and leaking. 

Tank = Till || jCeak 

Unfolding the meaning of the water tank into a hybrid action system we get 

Tank = \[var fill, y* : E,R 
fill, y := true, 0; 

alt fill A y > max — > fill := false 

with fill A y < max y = 1 © min < y y = — | 

]h 

Unfolding the linear composition yields 

Tank = |[ var fill, y* : E,M 

fill, y := true, 0; 
alt fill A y > max 
with fill A y < min 
1 f ill A rnin < y < rnax 

1 infill A min < y) V y > max > y = 



— > fill := false 

^ y = 1 
. _ 1 
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Finally, we translate the hybrid action system above into an ordinary action sys- 
tem. During the translation we use the properties for differential actions (Sec- 
tion 3.6) to simplify the result. The behaviour of the water tank is given by the 
action system 



var fill,y 


* : IB,R 










fill,y ;= 


true, 0; 










do f ill A 


y > max 


fill 


:= 


false 


(1) 


1 fill A 


y < min 


< 


min : y = 1 


(2) 


1 fill A 


min < y < max 


min 


< 


y < max : y = ^ 


(3) 


1 


A min < y 


min 


< 


1 

II 


(4) 



od 



When we analyse the behaviour of this action system, we notice that after 
initialisation only action (2) is enabled. This means that the tank first fills up 
to the min level at full rate. Then the action (3) becomes enabled; this action 
fills the tank up to the max level with half filling rate. When the tank is full 
action (1) becomes enabled; terminating filling. Immediately, action (4) becomes 
enabled letting the tank leak until the water goes below the min level. At this 
point, there are no more actions enabled, so the system terminates. 

A reusable filling system might be modelled by adding a control that resets 
fill. 



7.4 A step towards HyTech 

The action system for the water tank above can also be translated to the language 
of HyTech. The translation is merely a representation of the behaviour described 
above. As a first step we rewrite the verbal presentation into a hybrid automaton 
yielding Figure 3. In this automaton each state represents a differential action in 
the system. The boolean variable fill is implicitly encoded in the names of the 
states. The discrete actions are seen as transition actions in between the states, 
and hence the discrete action (1) vanishes from the resulting automaton. 




Fig. 3. A hybrid automaton describing the behaviour of the water tank. 



The automaton in Figure 3 is a linear hybrid automaton, and therefore it 
can be written in the language of HyTech. The following is the HyTech input 
for the water tank automaton. 
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— a water tank 

var y : analog; -- the water level 

automaton tank 
synclabs : ; 

initially fill & y=0; — initialization 

loc fill: while y<min wait {y'=l} — action (2) 

when True goto fill_leak; 

loc fill_leak: while min<=y & y<max wait {y'=l/2} — action (3) 

when True goto leak; 

loc leak: while min<=y wait {y'=-l/2} — action (1) , (4) 

end — tank 

This idea of translating a hybrid action system into HyTech works well, as 
long as the continuous evolutions are described by differential equations with 
constant right-hand side (Not to be confused with linear differential equations, 
where the right hand side is a linear expression in the variables) . This limitation 
comes from the theoretical model of linear hybrid automata, which is used as a 
basis in the tool. For the non-linear evolutions a linearisation must be performed 
before they can be expressed in the language of HyTech. 

8 Conclusions 

In this paper, we presented differential equations as primitive actions. Its weakest 
liberal precondition semantics was derived from the one for a discrete accumu- 
lation that approximates the continuous evolution. Differential actions provide 
a compact way of describing and analysing hybrid behaviours fusing standard 
mathematical analysis with discrete mathematics in the framework of weakest 
liberal preconditions as seen in the gate example. 

We also identified a class of hybrid action systems. They alternate between 
discrete actions and evolutions as typical for hybrid systems. Since the hybrid 
action systems are based on the use of differential actions where the time is 
implicitly present, we avoid the complication of a global time variable and an 
associated tick action. 

Parallel composition of action systems is extended to cover hybrid action 
systems. The extension is based on the mathematical properties of a linear com- 
position. Since the use of hybrid action systems and the extended parallel com- 
position does not impose any further constraints, all the existing results for 
action systems are directly usable. Moreover, the parallel composition between 
hybrid action systems and ordinary action systems remain meaningful with the 
extended parallel operator. 

The analysis of the hybrid behaviour in a hybrid action system is obtained by 
translating it into an ordinary action system. As illustrated with the water tank 
example, the translation is mechanical and based on the given general model. 
An existing tool, HyTech, can well be used for detailed analysis of a hybrid 
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action system where the evolutions are described by differential equations with 
constant right-hand side or by approximating, cf. Theorem 2. This possibility 
was also illustrated with the water tank example. 

These results are a foundation for further work. In particular, wc shall in- 
vestigate translations to HyTech of simple action systems. Furthermore, there is 
much remaining work in transferring results from control theory to the frame- 
work. Here, however, we consider it most fruitful to link this with specific classes 
of applications. A starting point could be a reformulation of Branicky’s classifi- 
cation. 

A very interesting idea is to study a variant of parallel composition, where 
the interaction between evolutions is through the differentiation variable. An 
example would be relative motion, where a person is walking with a speed ^ = 1 
relative to a conveyor belt, satisfying x = f{x). Here parallel composition would 
be conjunction of the two equations. This is already included in the current 
definition of parallel composition for a single evolution; but what will happen 
during transitions? 
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Abstract. This paper describes a methodology for the design of flight 
plans for rotary-wing unmanned aerial vehicles based on formal verifica- 
tion. The methodology uses linear hybrid automata to model the aircraft 
which will be used to perform a given mission, the flight plan that will be 
executed by that aircraft, the region where the flight will be performed 
and the meteorological conditions expected at the time the flight will 
be performed. The resulting model can be formally verihed with respec- 
t to previously established safety and timeliness requirements, like not 
running out of fuel or keeping minimum distances from ground during 
all phases of the mission. The result of this veriheation can be used to 
instantiate values in a parameterized flight plan or to assist an operator 
in incrementally constructing a flight plan whose feasibility can be guar- 
anteed in advance. The methodology is being embedded in a graphical 
flight plan editor which greatly reduces the time needed to plan a mission 
and increases the safety of the aircraft’s operation. 



1 Introduction 

Unmanned Aerial Vehicles (UAVs) have been successfully used for reconnais- 
sance purposes by military forces for over thirty years [Uni93]. More recently, 
the advantages of UAVs (low operational cost, no risk for human lives) have 
attracted the attention of civilian users and applications ranging from law en- 
forcement and ecological surveillance [Kan94] to high-altitude scientific research 
[NAS93] have been identified. 

Aircraft losses caused by human operator errors have traditionally been the 
biggest problem associated with UAVs [Ful96], having even determined the can- 
cellation of some promising UAV programs. Running out of fuel during a mission 
is the second most common cause of UAV losses, highlighting the complexity of 
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balancing efficient use of the aircraft in pursuit of the mission objectives with 
conservative, safe operation. 

Formal planning and verification of flight plans for UAVs can greatly increase 
the safety of operation of these aircraft, and reduce their operational cost by 
diminishing the workload imposed on the operator and therefore his/her training 
requirements. 

One of the ways to tackle the task of formally planning a mission for an 
unmanned aircraft is using hybrid automata^ a well suited approach for modeling 
and verifying hybrid systems. 

In this paper we describe how hybrid automata can be used to model a 
rotary-wing unmanned aerial vehicle and its operating environment. This model 
can be verified with respect to safety criteria, like not running out of fuel, and 
can also be used to perform parametric analysis, allowing the instantiation of 
symbolic parameters in flight plans for specific missions. 

Section 2 introduces aspects of rotary-wing unmanned aircraft relevant to 
this paper and describes the problem of generating flight plans for this class of 
aircraft. 

Section 3 is a short review of modeling and verification using hybrid automa- 
ta. 

The use of hybrid automata to model a rotary-wing unmanned aircraft and 
its operating environment is presented in section 4. 

In section 5 we show how the model developed can be used to verify a flight 
plan, i.e., to test if it is feasible, to instantiate a parameterized flight plan, i.e., 
to determine the boundaries of parameters in the flight plan so that it is feasible 
and to incrementally construct a flight plan whose feasibility can be guaranteed 
beforehand. 

Finally, conclusions are drawn in section 6. 

2 Mission Planning for Unmanned Aircraft 

2.1 Definitions 

We use the term mission to describe the operation of an aircraft in a given 
region, during a certain period of time while pursuing a specific objective. 

The ordered set of movements executed by the aircraft during a mission is 
defined in an associated flight plan. 

A flight plan can be decomposed in its component phases. Each phase is 
described either by the coordinates of a pair of way-points and by the speed at 
which the aircraft is to fly between these way-points or by its duration, an initial 
way-point and the speed of the aircraft. We consider a phase as completed when 
the second way-point is reached by the aircraft or, in the case of a specified 
duration, when the associated amount of time has elapsed. 

For the purpose of this paper, we introduce the concepts of a parameterized 
flight plan, a plan with at least one symbolic parameter, and flight plan instanti- 
ation, the process by which symbolic parameters in a parameterized flight plan 
arc instantiated to values for which specified requirements arc satisfied. 
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2.2 The mission planning problem 

Mission planning aims at producing a flight plan which allows the objectives of 
the mission to be pursued while complying with a set of mandatory and optional 
safety and timeliness requirements. 

We consider as internal conditions to be taken into account in the design 
of the flight plan the resources and characteristics of the aircraft used for the 
mission. External conditions are defined as the operational context, dictated 
by the region where the flight is being performed and by the meteorological 
conditions expected at the time of the flight. 



Internal conditions Each aircraft has a distinctive set of resources, which 
are used by the operator to pursue the mission objective. Resources have to be 
considered when planning a mission. They include the speed at which the aircraft 
can be operated, the fuel that can be carried on board, and electric energy which 
can be generated and/or stored on board the aircraft. 

Also, an aircraft is characterized by the specific fuel consumption at each 
speed, fuel tank capacity and how much electric energy is generated, stored and 
managed on board. 



External conditions To define the operational context of an unmanned air- 
craft we have to consider the prevailing meteorological conditions at the time 
the flight is performed. The aircraft’s dynamics arc affected by the speed and 
direction of the wind. This influence is most evident in small UAVs, which are 
often under-powered. 

A flight plan also has to consider the region were the flight will be performed, 
i.e., the elevations of the terrain and the existence of “no-fly” zones, zones were 
the operation of the aircraft is not allowed at all or during certain periods of 
time. 

The region were the flight is performed also influences the propagation of the 
radio waves used for communication between the aircraft and its ground control 
station. Particularly, the terrain profile can generate additional conditions when 
using line-of-sight communication links. 



Safety and timeliness requirements A flight plan has to take a set of safety 
requirements into account. Mandatory safety requirements, like not running out 
of fuel, have to be satisfied to avoid the loss of the aircraft. Optional safety 
requirements, like keeping mininium distances from the ground during the flight, 
increase the safety of the operation. 

Certain missions (mainly those involving search and interception) impose a 
time window on the opportunity to pursue an objective. Such missions have to 
satisfy not only safety requirements but also timeliness requirements. 
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3 Hybrid Automata: a short review 

The characteristics and the dynamics of an unmanned aerial vehicle can be 
modeled by hybrid automata. 

A hybrid automaton, as defined in [ACHH93,NOSY93,ACH+94], is con- 
structed by the generalization of a finite-state automaton, equipped with a set of 
continuous variables. A hybrid automaton is able to model discrete events and 
continuous activities, governed by a set of differential equations. 

Hybrid automata are described by a finite set of real- valued variables X and 
by a labeled multi-graph (V,E). The standard notation X is used to denote 
the first-order derivatives of X. The edges E represent the discrete events and 
are labeled with restrictions on the values of X before and after the execution 
of the corresponding actions. The vertices V represent the continuous activities 
and are labeled with restrictions on X and X during the corresponding activity. 
Therefore, the state of a hybrid automaton is modified by discrete events and 
by the passing of time. 

Formally, a hybrid automaton H = (X, V, E, syn, act, inv) is composed of: 

Variables A finite set X = {x\,X 2 , ■ ■ ■ ,x„) of real-valued variables. The size n 
of X is called the dimension of H . A valuation of // is a point s = (xq = 
0 . 1 , X 2 — 02 , ... ,Xn — On) in the n-dimensional real space i?" and represents 
the state of the continuous variables of H . We will use S to describe the set 
of all possible valuations of H . 

Locations A finite set V of vertices called control locations. A state of the 
automaton H is a. pair {v, s), where t> e V is a control location and s G i?" is 
a valuation. The term region is used to denote a set of states. The valuations 
associated with a location u in a region W are the valuations s such that 
(t>, s) G W. We will use X to denote the set of all possible states of the 
automaton H . 

Transitions A finite set of edges E called transitions. A transition e = (v, a, p,, 
v') consists of a source location v gV , a, target location v' gV , a synchro- 
nization label a G syn and a transition relation yiW . The transition e is 
enabled in a state {v, s) if for some valuation s' G S, (s, s') G /i. The state 
(v',s') is called a transition successor of the state (v,s). 

Synchronization Labels A hnite set syn of synchronization labels, used to de- 
fine the parallel composition of two automata: if two automata share the label 
a, then each o-transition in one automaton is followed by an o-transition in 
the other automaton. 

Activities A function act that assigns to each location v G V a .set of activities. 
Each activity is a function from the nonnegative reals to S. The model 
requires activities to be time-invariant: for all locations v G V, activities 
f G act(v), and non-negative reals t G i?-®, also (f + t) G act(v), where 
if + t){t') = f{t + t') for all t' G 

Invariants A function inv that assigns to each location v G V an invariant 
inv{v) D S. The automaton can stay in location v only while the invariant 
is true, that is, some discrete transition must be taken before the invariant 
becomes false. 
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Example 1. As an example, consider the hybrid system graphically represented 
in figure 1 (adapted from [ACHH93]). 

The system has one data variable x and two locations, Iq and l\. It starts in 
location ?o, where the value of x decreases at a constant rate of 1. The transition 
from Iq to li may be taken at any time after the value of x has fallen below 6, 
and it must be taken before the value of x falls below 5. When the transition is 
taken, the value of x is instantaneously decreased by 1. Once on location l\, the 
value of X increases at a rate between 1 and 2. The transition back to location 
Iq is taken exactly when the value of x hits 10. 




Fig. 1. Example of a hybrid automaton 



A hybrid system is described by a collection of hybrid automata, one for each 
component of the system. The constituent automata operate in a concurrent and 
coordinated way, sharing a set of common variables X , and synchronizing on the 
common set syni H syn .2 of synchronization labels. 

Current verification techniques only allow the analysis of linear hybrid au- 
tomata. A hybrid automaton is said to be linear if all its activities, invariants 
and transition relations can be described by linear expressions over the set of the 
automaton variables, X . This implies that for all locations v &V , the activities 
act{v) are defined by a set of differential equations of the form x = k^, one for 
each variable x € X, where kx G Z is an integer constant. 

There are various approximation techniques that allow a non-linear hybrid 
automaton to be converted in a linear one. For our purposes, the most useful of 
them is rate conversion, as described in [Ho95] and used in this paper. 

4 Modeling Flight Plans with Hybrid Automata 

As already mentioned, the problem of generating flight plans for UAVs can be 
tackled by hybrid automata. The position of the aircraft, the amount of fuel 
remaining on board the aircraft and the charge of the emergency accumulator 
are examples of continuous quantities while the transition from one phase of the 
flight plan to the next one is an example of a discrete transition of the same 
hybrid system. 

Existing algorithms only allow the verification of linear hybrid automata. 
We therefore consider the following hypothesis, which allow us to describe flight 
plans for unmanned aircraft by a set of linear hybrid automata [SF97]: 
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— Position and velocity of the aircraft are described in a tangent plane coor- 
dinate system, a Cartesian coordinate system whose origin is located at the 
takc-off/landing point. The error introduced by not taking into account the 
curvature of the Earth is negligible for the operational range of most UAVs 
(up to 500 km). 

— The transitions from one phase of the flight plan to the next are much shorter 
than the duration of the phases themselves, allowing them to be ignored. 

— The air-speed of the aircraft is constant during each phase of the flight. The 
speeds at which the aircraft can fly include best-range speed (the speed at 
which the distance traveled or area covered is maximized) , endurance speed 
(the speed at which airborne time is maximized), maximum speed and, for 
rotary-wing aircraft, low speeds used for hover. 

— The specific fuel consumption of the aircraft is not only a function of its speed 
but also a function of its mass, which decreases as the mission progresses and 
more fuel has been used. We use the rate conversion technique to linearize 
the specific fuel consumption of the aircraft. 

— The specific fuel consumption is also a function of the altitude at which the 
aircraft flies. The influence can be neglected for low altitudes or be considered 
at the expense of the additional modeling steps required to linearize the 
automata. 

— The terrain elevations, communications-covered regions and exclusion zones 
can be approximated with satisfactory resolution by a set of first-order in- 
equalities. 

Our approach to modeling an unmanned aircraft and its operating environ- 
ment consists of: 

— Modeling the dynamics of the aircraft under the influence of the prevailing 
(or expected) meteorological conditions with a hybrid automaton. We call 
this the X>-automaton. 

— Modeling the internal conditions (fuel consumption, electric charge of an 
emergency accumulator) with hybrid automata. We call this the T and Q- 
automata. 

— Modeling the external conditions (terrain elevation, “no-fly” zones, radio- 
waves propagation) with inequalities over the tangent-plane coordinate sys- 
tem. 



4.1 Aircraft dynamics 

The aircraft’s dynamics, i.e., its position and velocity as a function of time, 
can be modeled by a linear hybrid automaton in which each control location 
corresponds to one of the phases of the flight plan. 

The activities of each control location contain functions which describe the 
evolution of the aircraft’s position on a tangent plane coordinate system. Remem- 
ber that, according to our previous assumption, the speed during each phase of 
the flight is constant. 
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The transitions connecting the different control locations are guarded by tests 
belonging to one of two categories: 

— Tests on the aircraft’s position. This type of test is used when the end of 
the n-th flight phase is defined by passing way-point WP„+i = (x„+i, yn+i, 
Zn+i), specified by coordinates in the tangent plane coordinate system. 

— Tests on a clock. This type of test is used when the duration of the n-th 
phase of the flight plan is specified, instead of its final way-point. The clock 
used has to be initialized when entering control location n. 



Example 2. Consider a partial flight plan with only three phases: 

1. Flight at best-range speed at a constant altitude of x meters, from way-point 
WPi = (xi, yi, z) to way-point WP 2 = (x2, V2, z). 

2. Hover at WP 2 = (x 2 ,y 2 ,z) for t 2 seconds. 

3. Return to WPi = {xi,yi,z) at maximum speed. 

Figure 2 shows a linear hybrid automaton which models the dynamics of the 
partial flight plan just described. 



xi^yi, z jx < X2 A y < V2 
X = Vj 
y = Vr 
i = 0 



hover 

X = X2 Ay = y2 
th ■■= 0 




Fig. 2. A linear hybrid automaton used to model aircraft dynamics 



The automaton starts at location PI. At this location, the aircraft’s position 
in the tangent plane coordinate system is described by the differential equations 
X = Vr^ and y — and Vr^ are the north and east components of the 

aircraft’s best-range speed Vj. after considering the influence of wind: 

COS ‘Tp + Vw cos tpw 
Vry = Vr sin Ip + Vyj sin ipy, 
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where = arctan( {y 2 <8) yi)/{x 2 ® x\) ) is the course that leads from WPi to 
WP 2 , V'u) is the direction to where the wind blows and Vyg is the wind’s speed. 
It is important to note that the model accepts constant values for the wind’s 
speed and direction as well as ranges of minimum/maximum values. 

At WP 2 = (x 2 ,y 2 ,-z), the transition that leads to P2 is enabled and the 
invariant of PI turns false. Firing the transition sets clock th '■= 0, which will be 
used to time the duration of the hover. The synchronization label hover is used 
to synchronize other automata used to model the aircraft. 

The automaton remains at location P2 for t 2 seconds. Note that, during the 
hover, x = y = 0. After t 2 seconds have elapsed, the automaton progresses 
to location P3, which is used to model the return to WPi at maximum speed 
{maxspeed) . 

4.2 Specific fuel consumption 

In general, the specific fuel consumption of a rotary-wing unmanned aircraft is 
a function of several variables and flight conditions which makes the resulting 
system a nonlinear hybrid one. In order to be analyzed, this system must be ap- 
proximately modeled by a linear hybrid automaton. This is accomplished by the 
rate translation technique [Ho95] , which replaces the nonlinear fuel consumption 
variable by a piecewise-linear variable that approximates the original one. 

Let us consider, for example, a model for the specific fuel consumption (s/c) 
which reflects its dependency on the aircraft’s mass and flight condition (hover, 
best range speed, endurance speed, maximum speed). Thus, in general, we have 

sfc = f = 9u{m) 

where m = me + / is the total aircraft’s mass with mg being the empty aircraft’s 
mass, / the current fuel mass on board and u e {hover, hestrange, endurance, 
maxspeed} a parameter determining the flight condition. Under such conditions, 
Pu is obviously a non-positive function which decreases when m grows. 

Let us assume as being approximated by 

guim) = gu{me) + h„/ 

where gu{me) is the nominal speeifie fuel consumption for flight condition u, and 
hyf its increment due to the current amount of fuel / in the aircraft, hy being 
a negative constant. 

Considering a flight plan as the composition of several phases, each of them 
corresponding to a constant flight condition, we could model the fuel consump- 
tion of a rotary-wing unmanned aircraft by a nonlinear hybrid automaton such 
that location Pj models a phase with fuel consumption dynamics given by 

sfc = fcp + k 2 j 

with kl^, k 2 i being non-positive constants corresponding to < 7 «(me) and for 
location i and flight condition u. Also, at each location Pi, / is constrained to be 
in the interval [0, P] where F is the initial amount of fuel on board the aircraft. 
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A particTilar ND-FUEL location with dynamics / = 0 is used to model the 
situation in which the aircrafts runs out of fuel. Transitions in the automaton 
arc either forced by synchronization labels representing changes in the flight 
conditions or by guarded transitions / = 0 which test the no-fucl condition and 
lead the automaton to the NO-FUEL location. 

The resulting nonlinear hybrid automaton is bounded [Ho95] and can be 
translated into a linear hybrid automaton by approximating the nonlinear vari- 
able / by a piecewise-linear variable. A possible simple linear approximation is 
the one obtained by replacing a location Pi with a location satisfying 

0 <= / <= Fi 

and 

/ G [fcp -I- 

where Fi is calculated from a lower bound estimate of the fuel burned in the 
previous phases. 

Figure 3 shows a linearized automaton which models the fuel consumption 
of an unmanned aircraft executing the partial flight plan shown in example 2. 




Fig. 3. A linearized hybrid automaton used to model the specific fuel consump- 
tion of a rotary-wing UAV 




Hybrid Automata for the Mission Planning of Unmanned Aerial Vehicles 333 

4.3 Emergency accumulator 

The charge left on an emergency accumulator can be modeled by the automaton 
shown in figure 4. The location CHARGE is used to model the charging of the 
accumulator with a constant current Ichg- Assuming that the aircraft has some 
kind of energy management capabilities, discharging the accumulator is modeled 
by the locations DISCHARGEl (where the current drained from the accumulator is 
Idis) and DISCHARGE2 (where the current drained is I min < Idis)- Two additional 
locations, FULL and EMPTY, are used to model the fact that the charge of an 
accumulator cannot be greater than its nominal capacity or lesser than zero. 



discharge 




Fig. 4. A linear hybrid automaton used to model an emergency accumulator 



Labels are used to synchronize this automaton with the rest of the model, so 
that the energy used during the initialization of the aircraft (when the engine 
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is not running and therefore the generator is not produeing any energy) can be 
considered and failures of the generator /power conditioner can be modeled. 

4.4 Communication coverage, terrain elevation and “no-fly” zones 

We model communication coverage by a region, described by an inequality, where 
the signal is strong enough to guarantee the communication with the ground 
control station. This potentially non-linear region can then be approximated by 
the union of several linear regions. 

Example 3. Assuming an omni-directional antenna and homogeneous propaga- 
tion of radio-frequency waves in a given terrain, the coverage zone is described 
by a circle of radius Rmax- Let us assume that the antenna is located at the 
takeoff/landing point, the origin of the tangent plane coordinate system. We 
therefore have that a strong enough signal in the region described by following 
inequality: 



x^ + y^n 

This non-linear region can be conservatively approximated by the union of 
several linear regions: 
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Figure 5 illustrates graphically the approximation of -f n R^ax Ly the 
set of regions presented above. 

In a similar way, terrain elevation and “no-fly” zones are modeled by three- 
dimensional regions. 

4.5 Meteorological conditions 

The influence of wind has been already described when dealing with the aircraft 
dynamics (section 4.1). 

5 Design of Flight Plans with Hybrid Antomata 



In the following we present different ways to design flight plans for unmanned 
aerial vehicles based on formal verification of hybrid automata. 
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Fig. 5. Modeling of communications coverage 



A methodology for the analysis of linear hybrid automata is presented in 
[ACH+94,AHH96,Yov93]. The methodology is based on predicate transformers 
for computing the step predecessors and step successors of a given set of states. 

Using successive approximation, the methodology allows the eomputation of 
post*{W), the reachable region of a region W, i.e., the set of all states that are 
reachable from states in W. Conversely, it is possible to compute pre*{W), the 
initial region of W , the set of all states from which a state in W is reachable. 

The more general problem of verifying if a hybrid automaton satisfies a re- 
quirement expressed in a formula of the real-time logic TCTL (timed computa- 
tion tree logic) is also addressed in [ACH+94]. 

The hybrid automata modeling the mission of an unmanned aircraft can be 
formally verified with respect to safety and timeliness criteria. We call this formal 
verification of the flight plan associated with the mission. In this paper, we will 
only discuss the verification of safety requirements. 

One of the ways of specifying safety criteria is to use a real-time temporal logic 
like TCTL. By using the operators never, always and eventually, it is possible 
to describe the desired behavior of the hybrid automaton being verified. 

Another way of specifying safety criteria is by using the concept of regions, 
sets of states of the hybrid automaton being investigated. All the safety criteria 
that flight plans for UAVs have to satisfy can be expressed in terms of inclusion 
in a “good” region and intersection with a “bad” region. 

We use regions to specify safety criteria for mission planning since three- 
dimensional regions on the tangent-plane coordinate system seem very natural 
to our problem. 
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Flight plans can be designed in three different ways: 

— The feasibility of a proposed flight plan can be verified. The result of the 
verification process is either “flight plan is feasible” or “flight plan is not 
feasible” . 

— Symbolic parameters in a parameterized flight plan can be instantiated so 
that the resulting flight plan is feasible. The result of the instantiation pro- 
cess is “flight plan will be feasible if . 

— Flight plans can be incrementally constructed by computing the safely reach- 
able region as each phase is added to the partially constructed flight plan. 
The overall feasibility of a flight plan constructed in this manner is therefore 
guaranteed in advance. 

5.1 Verification of flight plans 

Using regions, the problem of verifying the feasibility of a flight plan can be 
summarized as follows: 

1. Construct an automaton that describes the mission (the Al-automaton). 
This is the parallel composition of the T), T and Q-automata. 

2. Starting with initial conditions at the take-off point (region /), compute the 
forward reachable region of the Al-automaton. 

3. Compute the intersection of the reachable region with the “bad” region (de- 
fined by the external conditions in the form of inequalities on the tangent 
plane coordinate system). If the intersection is not empty, the feasibility of 
the flight plan cannot be guaranteed. 

4. Verify if the reachable region is contained within the “good” region. If not, 
the feasibility of the flight plan cannot be guaranteed. 

Tools for the automatic analysis of hybrid systems, like HyTech (described 
in [HHWT95,HH95]), can be used to perform the parallel composition of hybrid 
automata and to compute the reachable region of the resulting hybrid system. 
HyTech is also able to manipulate regions, computing their intersection and 
verifying if a region is included within another one. 

5.2 Instantiation of parameterized flight plans 

The methodology for the analysis of hybrid systems presented in [ACH+94] is 
able to manipulate not only numerical quantities but also symbolic parameters. 

We can use symbolic parameters to instantiate parameterized flight plans, 
i.e., determine for which values of the symbolic parameters the safety require- 
ments are satisfied. Instantiation can be summarized as follows: 

1. Construct an automaton that describes the mission (the Af-automaton). 
This is the parallel composition of the T>, T and Q-automata. 

2. Compute the backward reachable region of the A4-automaton, starting at 
the overall “bad” region. 
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3. Intersect the result with the initial region I and instantiate all symbolic 
parameters. The values obtained at this step are those that will lead from 
the initial region to the “bad” region. 

4. Complement the parameters obtained in the previous step. These are the 
values for which the mission is feasible. 

5.3 Incremental construction of flight plans 

Incrementally constructing a flight plan can be done by continuously computing 
and displaying the reachable region of the A4-automaton over a map of the 
terrain where the flight will be performed, as shown in figure 6. 

The consequences of adding or removing way-points to the flight plan are 
immediately reflected in the shape of the reachable region, assisting the operator 
in the determination of the next phase of the flight plan being constructed. 

Further details of such a graphical flight plan editor with an underlying ver- 
ification engine can be found in [SFCD98] . 




Fig. 6. Incremental construction of flight plans 
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5.4 Example of flight plan verification 

Let us consider the sample mission shown in figure 7. The mission consists of 
over-flying a couple of islands off the coast of Santa Catarina in the south- 
ern part of Brazil while investigating outlawed fishing activities on the islands. 
Take-off/landing point coordinates are 27°40.9' S, 48°33.8' W. Coordinates of 
Island Francisca are 27° 42. 2' S, 48°33.9' W and of Island do Largo 27°42.4' S, 
48°35.6' W. Tangent plane coordinates are, respectively, (0,0), ((8!2247, ;8)118) 
and (®3051, (8>3080)^. The aircraft will be flown at its bestrange speed, 15 m/s. 
Specific fuel consumption is less than 0.56 g/s under such conditions. 




We want to verify if an initial amount of fuel F — 1000 g is sufficient to ac- 
complish the mission while avoiding the approach corridor of the nearby airfield, 
described (in tangent plane coordinates) by 

®2/5y 4- 700 < x < {g)2/5y + 3700 

Using HyTech we find that the reachable region of the A4-automaton is 

post* {I) = 0 = a; 4- 15t f\ Ax = 75y A 27/ — x + 27000 A 
0>=x A x + 2250 >=0 

V 0 = X 4- 4t 4- 1650 A 8y = 29x 4- 64290 A 36/ = 5x 4- 44250 A 
0 >= X + 2250 A X 4- 3045 >= 0 

V 32t = 3x + 20295 A 63x = 6Ay + 285 A 5x + 96/ = 62175 A 

^ tangent plane coordinates arc expressed in meters 
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X + 3045 >= 0 A 0 >= X 

V X = 0 A 64y + 285 = 0 A 32/ = 20725 A 32t >= 20295 

We can easily verify from the above result that the aircraft uses g of fuel 
(which is less than F = 1000 g) and that the intersection of the reachable region 
of the A4-automaton and the “no-hy” zone defined by the approach eorridor to 
the airfield is empty. We can therefore conclude that the proposed flight plan is 
feasible. 

6 Conclusions and Further Work 

Hybrid automata are well suited to model unmanned aircraft and their operating 
environment and the usage of regions is very natural for the verification of safety 
requirements. 

We are generating flight plans for the Helix UAV [Ltd92], shown in figure 8, 
with a prototype mission planning tool that implements the verification, instan- 
tiation and incremental construction of flight plans techniques described in this 
article. Our experience is that the use of this techniques greatly increases the 
confidence of the operator, even if working around some overflow problems of 
HyTech very often requires manual intervention in the generated models. 




Fig. 8. UAV for which a prototype mission planning tool is being used 



In the near future we intend to use the methodology not only for mission 
planning (construction and verification of flight plans prior to their execution) 
but also for mission control (during the execution of the mission). In this sit- 
uation, a model will be constructed and verified on-line periodically, providing 
an additional level of supervision when the aircraft is operated in an interactive 
mode. 
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Abstract. This paper presents a solution to certain problems in 
switched controller design for stochastic dynamical systems with a qua- 
dratic cost. The main result is a separation theorem for partial informa- 
tion systems. This result is then used to convert the partial information 
stochastic control problem to a complete information stochastic control 
problem. We also show that certainty equivalence does not hold. The 
optimal sequence of controllers can be determined via an appropriate 
solution to a dynamic programming problem. 



1 Introduction 

There are many real industrial control problems where the control action is 
determined by switching among a finite number of given control laws. Typi- 
cal examples include an automobile transmission, where different gears need to 
be switched in order to obtain desirable performance (gain switching) [1,2], a 
thermostat where the plant heats up when the control is on and cools when the 
control is off, and steering an automobile where switching between appropriately 
chosen controls laws can make an otherwise uncontrollable vehicle controllable 
and allow the vehicle to be steered (locally) in any desired direction [3](pg 76). 

In [4,5] we devised optimal strategies for switching between given control- 
lers at fixed switching intervals for plants described by an uncertain differential 
equation. In this paper we consider optimal switching strategies for plants de- 
scribed by linear time-varying stochastic differential equations with a quadratic 

* This work was supported by the Center for Sensor Signal and Information Processing 
the Australian Research Council and by US Army Research Office under the MURI 
grant “An Integrated Approach to Intelligent Systems”, grant DAAH04-96- 1-0341. 
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cost functional and develop optimal switching strategies for both the full and 
partial information cases [6,7]. We consider separation and certainty equivalence 
for switched systems. Although these notions have been studied for many years 
[8] and their applicability to various classes of stochastic systems continues to 
be of great interest e.g. [9]. To our knowledge these important notions have not 
been investigated for linear switched controller and hybrid control systems. This 
paper presumes a background knowledge of linear stochastic control systems and 
Ito calculus. An excellent treatment of these subjects can be found in [10] and 
[ 11 ]. 

In this paper we show that the full information problem with quadratic cost 
the optimal switching sequence of controllers is given by a solution to a dynamic 
programming equation. We show that where the switching function depends on 
both time and state (state estimate) certainty equivalence does not hold. For 
the partial information case a separation theorem is established. This is then 
used to convert the partial information problem to a full information problem 
enabling the optimal controller switching sequence to be determined. 

The organization of this paper is as follows. Section 2 considers full infor- 
mation switched controller systems. The main results of this section is to show 
equivalence between the optimal controller switching sequence and the solution 
to a dynamic programming equation. In this section it is also shown that for the 
class of systems considered in this paper certainty equivalence does not hold. 
In section 3 the partial information problem is considered. The main results of 
this section are given in terms of the existence of suitable solutions to a Ricatti 
differential equation of the Kalman Filtering type and appropriate solutions to a 
dynamic programming equation. In section 4 a simulation example is presented. 

2 Optimal Controller Switching with Full Information 

All random variable in this paper are defined on the probability space (17, A, V). 
Let w;(r) be a vector valued Wiener process with covariance matrix Q(f). Let £ 
denote expectation with respect to the measure V. Let N S be a given pos- 
itive integer. The permissible switching times ■ ■ ■ An} corresponding 

to an ordered discrete set where {ti G R+). 

Consider the following stochastic system whose dynamics on the interval 
[Ojfjv) are given by the stochastic differential equation 

dx[t) = (A{t)x{t) + B 2 {t)u{t))dt + Di{t)dw{t)\ x'(0) = xq (1) 

where x[t) G R" denotes the state, u{t) G R^ is the control and the functions 
A(t), B 2 {t), C(t) are given piecewise continuous matrix functions of time 

of appropriate dimension. 

Controller Switching with State Feedback [12,5,4,13]. Given a finite collection of 
controllers 



Ui{t) = Ui{t,x{t)), U2{t) = U2{t,x{t)), 



Uk{t) = Uk{t,x{t)) ( 2 ) 
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where the control laws U 2 {Qf), • • • , 17fc(Qf) are given continnons matrix 

functions. Let Ij{f) denote a state switching function which is permitted to 
switch between controllers at set switching times tj, {j — 1, . . . , N}. The class of 
switching functions considered in this section is of the form /j(f) : lo } ^ 

{1, 2, . . . , k}, which map from the set of state measurements {x(f) } to the set 

of symbols {1, 2, . . . , fc} which index the controllers (2). This class of switching 
functions defines a family of dynamic nonlinear state feedback controllers of the 
form: 



Vj G {0, (t,x(t)) Vt G where ij ^ |g"). 

( 3 ) 

Hence the control strategy, which is a rule for switching from one controller 
to another, constructs a symbolic sequence from the set of state mea- 

surements x{f) Iq^. The set of all admissible controls achievable by controller 
switching (3) with the controllers (2) is denoted by U. 

Let 5(1) be a given function which maps from i?" to R, let N{t) and M{t) 
be given matrices of time satisfying N{t),M{t) n pi, p > 0 Vf G [0,t]v]- Xf H 0 
is a given constant matrix which penalises the final state. 

Introduce the following quadratic cost function. 



W {t, X, u{t)) = 8 



X {t)N{t)x{t) +u {t)M{t)u{t) 



( 4 ) 



and define 

f;(xo, 5(0) [5(x(t,-+i))] + f W{t,x,u^{t))dt. (5) 

Jt, 

Definition 1 fState Feedback Stochastic Control Problem^. The state feedback 
stochastic control problem with system (1) is said to have a solution via controller 
switching with the controllers (2) if the following conditions hold: 

(i) For any admissible control sequence, u &IA, there exists a unique trajectory 
of system (1) on the interval [0, tAr). 

(a) The cost 

Jiu)ueu=£(^j x\t)N{t)x{t) + u'{t)M{t)u{t)dt + x'tj^XfXt^'^ (6) 

exists and is finite for some admissible control sequence, u a U. 



Theorem 1. The state feedback stochastic control problem (Definition (1)) for 
system (1) has a solution via controller switching with the state feedback con- 
trollers (2) if and only if the dynamic programming problem 

Vn{xo) = £ [x^^XfXt^] ; Vj{xo) = .jnin^f [F({xq, P,-+i(r))] (7) 

has a solution for all j = 0,1, N ^ 1 and all xq G R". 
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Furtherm,ore let ij(xo) he an index such that the minim, um m (7) is achieved 
fori — ij(xo). Then the controller (2), (3) associated with the switching sequence 

{'h}f=^o^ where ij = ij{x{tj)) solves the state feedback stochastic control problem. 
Proof. Sufficiency. For any j = 0,1, ..., N introduce the cost to go 

Vj = min £ \ [ X {t)N{t)x{t) +u {t)M[t)u{t)dt -\- x^ X jXiJ\ . (8) 

u(t)€U j 

Following [14,10] Vj{f) satisfies (7). Also it follows from (8), that if the stochastic 
control problem has a solution, then Vo(xo) exists and is finite. 

Necessity. Equation (7) implies that for the controller associated with the 
switching sequence we have 

8 X {t)N{t)x{t) +u (f)M(f)u(f)df + n Vb(x(0)). (9) 

Since Vb(x(0)) = J{u*,x{0)) fl x(0)), the controller associated with 

the switching sequence, {ij}fTQ^, solves the state feedback stochastic control 
problem. 

Suppose that the given state feedback controllers (2) are of the form 

ui{t) ^ Ki{t)x{t), U 2 {t) ^ K 2 {t)x{t), ..., Uk{t) ^ Kk{t)x{f) (10) 

where Ki{t) are given matrices of time, then we have. 

Definition 2 Certainty Equivalence [10,6]. The sy.stem (1) is said to satisfy a 
certainty equivalence principle if the optimal admissible control law is indepen- 
dent of the statistics of the noise inputs. 



Corollary 1 Consider system (1) and state feedback controllers (10). For this 
class of stochastic control problems certainty equivalence does not hold. 



Proof. Consider system (1) with the state feedback basic controllers (10). All 
that is required to be shown is that the controller switching sequence for the 
plant with no noise is different to that with noise. 

The cost J (u) for all admissible controllers can be rewritten as 



duCilAiF) — 8 X^^ X gXt 



Expressing the state as 




x\t){N{t) + Kl^R{t)K,,)x 




( 11 ) 



x{t) = x{t) + x*{t) 



(12) 



with 



x{t) = (A + B 2 Kifjx{f), a;(0) = x(0) 
dx*(f) = (A + B 2 Ki^)x* [t)dt + dw{t) x*{0) = 0, 



(13) 

(14) 
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it follows that 

Ex*{t) — 0, Ex {t)x*{t) — x{t)Ex*{t) — 0 Vt. (15) 

Following standard dynamic programming we substitute of (14) and (15) into 
(11) and rewriting (11) as: 




It can now be seen that in general the choice of gain matrix that optimizes the 
second and third terms in the above expression (16) will depend on both the 
state as well as the noise covariance and its contribution to the state over the 
interval [t]v-i,^iv)- This contradicts certainty equivalence. 



Remark In switched controller systems the optimal controller switching sequence 
for linear feedback controllers depends on the input noise covariance. This result, 
also applies to zero-mean white Gaussian process noise inputs. This differs from 
optimal linear quadratic Gaussian control [7,6,15] were the optimal feedback 
gain is independent of the input noise covariance. 

We illustrate that Corollary (1) above by considering the system (1) with 
parameters (29) in the simple scenario corresponding to only one switching time 
with switching occurring at T = 0. Let the system be defined over the interval 
[0, 0.1) with Xf = 0. The process noise w{t) is zero mean Gaussian with covari- 
ance E{w{t)w (t)) = 1.0. For this plant the cost (when there is no process noise 
is given by the second component of (16)) and is equal to J{ui) = XqZiXq for 
controller u\ and J{u 2 ) = x'qZ 2 Xq for controller U 2 where 

_ To. 5159 00.0314] _ (0.5701 00.0873' 

"^^“[00.0314 0.3257 J “ [00.0873 0.3459 

When there is noise the cost of using any of the given controllers is determined by 
the second and third components of (16) and is equal to J{ui) = XqZiXo + 0.0152 
or J{u 2 ) = Xg02a^o + O.O21O. Consider the point x = [0.1, 0.1] for the case without 
any noise, U 2 is the optimal control to use, whereas with noise the optimal control 
to use is u\. 

It can be clearly seen from Figures (1) and (2) that the controller switching 
regions depend on the process noise covariance. This differs from standard L.Q.G. 
optimal control were the controller gain matrix is independent of the input noise 
covariance for zero-mean Gaussian process noise. 
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3 Optimal Controller Switching with Partial Information 

Let be a random variable with Gaussian probability law, mean xq and co- 
variance Fo and t>(^ is a Wiener processes independent of w(^ with covariance 
matrix F(^. Consider the following stochastic system whose dynamics on the 
interval [0, tAr) are described by the stochastic differential equations 

dx{t) = {A{t)x{t) + B 2 {t)u{t))dt + Bi{t)dw{t), x(0) = 

dy{t) — C{t)x{t)dt + dv{t), (17) 

where x{t) G R" denotes the state, y{t) G denotes the output and u{t) G R^ 
is the control input. The functions A{t), Bi{t), B 2 {t), C{t) are known given 
piecewise continuous matrix functions of time. We assume that R{t) G pi, p > 
0 Vi G [0,tjv), and we define the following two families of sigma algebras 

J’* = a{^,w{s),y{s),s G t) 

Z* = a{y{s),s € t) (18) 

Controller Switching with Output Feedback [12,5,4,13]. Given a collection of out- 
put feedback controllers 

ui{t) ^ Ki{t)y{t), U 2 {t) ^ K 2 {t)y{t) , ..., Uk{t) ^ Kk{t)y{t) (19) 

where 7Li((), K 2 {'^, ■ ■ ■ , Kk{^ are given continuous matrix functions of time, we 
define a switching function which is permitted to switch the controllers (19) 
at switching times tj {j = 1, . . . , A^}. The class of output switching functions 
considered in this section is of the form : {Z*^} —> {1.2,...,fc}, which 

map from the output sigma algebra {Z^^} to the set of symbols {l,2,...,fc} 
which index controllers (19). This class of switching functions defines the family 
of dynamic nonlinear output feedback controllers of the form: 

Vj G {0, 1, . . . , iV G 1} u{t) = Ki-{t),y{t) yt e [tj,tj+i) where ij = <I>j{Z^^). 

(20) 

The control strategy now becomes a rule for switching from one controller to 
another to construct a symbolic sequence from the sigma algebra Z^j . 

Furthermore let X denote the set of all controls achievable by controller switching 
(20) with the controllers (19) adapted to Z^ . 

The solution to the partial information stochastic control problem considered 
in this section involves the following Ricatti differential equation 

P{t) = P{t)A'{t) + A{t)P{t) 0 P{t)C'{t)R{t)-^{t)C{t)P{t) + B2{t)Q{t)B'^{t) 
where P{0) — Pq (21) 

We consider the estimator equations of the form 

x{t) = A{t)x + P{t)C {t)R{t)^^{y{t) 0 C{t)x) where x(0) = ^ (22) 

which correspond to the continuous time Kalman filter [16]. 
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Let be a given fnnction which maps from 7?" to R and let xq S be a 
given vector. Introduce the following cost function: 

Wi{t, X, u{t)) = £ (^ {t)N{t)x{t) + u {t)M {t)u{t)^ . (23) 

Then 



F^{xo,S{^) = £ {S{x{tj+i))) + f Wi{t,x,Ui{t),y{t))dt. (24) 

Jtj 

Definition 3 The output feedback stochastic control problem with system (17) 
is said to have a solution via controller switching with the controllers (19) if the 
following conditions hold: 

(i) For any admissible control sequence, u G X, and initial condition x(0) there 
exists a unique trajectory to the system state on the interval [0,tAf) and the 
solution to the Ricatti differential equation (21) exists and is positive definite 
for all t G [0, tjy). 

(a) The cost given by 

J{u)ue:x = £ (^J X {t)N{t)x{t) + u {t)M{t)u{t)dt + x^^XfXtJ^'^ (25) 
exists and is finite for some admissible control sequence, u G X , 



Theorem 2. The output feedback stochastic control problem (Definition (3)) 
has a solution via controller switching with the output feedback controllers (19) 
if and only if the solution P($) to the Ricatti equation (21) with initial condition 
P{0) — Pq is defined and positive definite on the interval [0, tjy) and the dynamic 
programming equation 

Vn{xln) = ^ [x[^XfXt^) ; Vj(i(^) = (P)(it^ , J)) (26) 

has a solution for j = 0,1, . . . , N (S> 1 for all xq G R". 

Furthermore let ij{xo) be an index such that the minimum in (26) is achieved 
for i = ij{xo) and x(?) be the solution to the equation (22) with initial condition 
£•(0) = xq. Then the controller (19), (20) associated with the switching sequence 

where ij = <Pj{x{tj)) solves the output feedback control problem. 

Proof. {Outline) By virtue of Girsanov’s theorem, Lemmata (1,2, 3, 4) in Section 
6 below and Lemma (2.4.3) from [6] it follows that the evolution of the state 
of (17) with u{t) G 0, denoted a{t) satisfies £'^[a{t)\Z^] — f“[o:(t)|P^]. Thus 
the states estimates generated by a Kalman filter are optimal when there is 
controller switching as considered in this paper. This is established in detail in 
section 6. It then follows that under the new probability measure (defined in 
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section 6), the estimation error, e = x{t) 0 x(t) is a Gaussian process with zero 
mean and covariance matrix given P{t) by (21). Hence, if the output feedback 
stochastic control problem has a solution via controller switching it follows that 
the solution, F((), to the Ricatti differential equation (21) with initial condition, 
F(0) = Pq, is defined and positive definite on Now from Lemma (4) we 

have that the minimization of cost (25) is equivalent to minimizing (26) where 
X is the estimator state, R(x(tjv) = £'^x{tpf)X fx{t]\r) is the cost at the final 
position and 



dx{t) = {A{t)x + B 2 {t)u{t))dt + P{t)C 0 C{t)x)dt 





= {A{t)x + B 2 {t)u{t))dt + du{t) 


(27) 


where 


£^Ht)] = 0 






£^[iy' {t)iy{t)] = P' {t)C {t)R{t)-^C{t)P{t). 


(28) 



The statement of the Theorem now follows immediately from Theorem 1. 

Remark Results for discrete time switched controller systems can be shown to 
trivially follow from the results presented in this paper. 



4 Illustrative Example 



We consider the 2 dimensional, unstable, non-minimum phase system 



A{t) = 



0 r 

01.25 1 




'O' 

1 




"0" 

_i! 


,C{t) = 


‘0i‘ 

2 



(29) 



Consider the system (17) the following set of output feedback controllers 

ui(f) = [01, 2]f(<), U2(f) = [3, 06]i(t), (30) 



and permissible controller switching at time instants jT, j = 0, 1, . . . , 250, T = 
0.05 over the time interval [0,12.5) seconds. It can be easily verified that each of 
the controllers in (30) is independently unable to stabilize the plant. 

For simulation purposes set to Xf = 10/ and the matrix observer Ricatti 
equation (21) initial condition F(0) = I. The process and observation noises were 
also set to £{w{t)) = 0, £{w{t)w (t)) = 0.1, £(iy(t)) = 0 and £(iy(t)iy (t)) = 0.1. 
The matrices X(t) and M{t) are N{t) = 0.1 and 



M{t) 



5 0 
0 2 



(31) 



Using the results of Theorem 2 we see in Figure (3) that by switching the two 
unstable controllers the system state remains within an a small neighborhood 
about the origin. The control input to the system, is shown in Figure (4). The 
switching between the two controllers is evident. 
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5 Conclusions 

In this paper we presented a solution to certain problems in switched controller 
design for stochastic dynamical systems. The main result is a separation theorem 
for partial information systems which can be used to convert the partial infor- 
mation stochastic control problem to a complete information stochastic control 
problem. We also show that certainty equivalence does not hold and for different 
noise statistics the optimal switching sequence changes. Finally we show that 
the optimal sequence of controllers can be determined by solving a dynamic pro- 
gramming equation. The results presented in this paper can be extended to the 
infinite time interval after some technical issues are taken into consideration. 



6 Appendix 



There is an intrinsic difficulty in output feedback stochastic control problem- 
s arising from the fact that the controller and hence the control depends on 
the observation. This difficulty has been solved for linear quadratic Gaussian s- 
tochastic control problem but it is not immediately clear that these results cover 
the class of switching systems considered in this paper. Thus in order to establish 
Theorem (2), we need to establish the existence of a new probability measure 
in which the output and all the admissible controls are independent. This then 
overcomes the difficulty described above. In order to establish the existence of 
such a measure we will need the following definitions and Lemmata. The argu- 
ment here essentially follows [6] except that special attention must be given to 
the switching times were solutions to the equations describing the dynamics are 
not well defined. 

Define the processes a(^, /3(^ by 



da — A{t)adt + dw{t), a(0) = ^ 
d[3 — C{t)(3dt + dv{t), /3(0) = 0 



also define Xi, j/i by 



(32) 

(33) 



dxi = {A{t)xi + B 2 {t)u{t))dt, xi(0) = 0 
dyi = C{t)xidt, yi(0) = 0. 

For any admissible control u{t) we define 

x{t) — a{t) + Xi{t) 
y{t) = a{t) -h yi{t). 

We then consider the following two processes [6] 



L40 



? 7 “(t) = exp 
d^v{t) = y[t) ® / C{t)x{s)ds 



x' [t)C' (t)R{t) ^dy®— / x{t)' C [t)' R{t) ^C[t)x{t)ds 

2 ./n 



(34) 

(35) 



(36) 

(37) 

(38) 
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where the superscript u is to show the dependence of on the the control u{t). 

The process rf' (38) satisfies the stochastic differential equation for all time 
t G [0, ^ tj where tj are the controller switching times 

drf = rj'^{x'C')R-^dy, t7 "(0) = 1. (39) 

Furthermore we define J*{u) as 



£ 



(40) 



Hn 



V^{iN) {^n)^ fx{iN) + J r]^{t)x {t)N{t)x{t) + u{t) M{t)u{t) j dt 

Since the controls arc piecewise linear (affine in the state) between the switching 
times on the interval [tj, tj+i) all that is required to be shown for the existence of 
a new probability measure which makes the observation process independent 
of the control are that the cost, under the new measure is finite and 

that the Radon-Nikodym derivative [11] 



~dP 



?'/“(^w) 



(41) 



is a martingale. This is established in Lemmata (1,2,3) below which closely 
follow [6] but with modifications required to account for controller switching on 
a countable set. 



Lemma 1. There exists a constant, Ci, independent of the control such that 

fry“(t) I a{t) |^G Ci (42) 



Proof. Consider the following approximation of 77", 



rif{t) = exp 



x[C'{t)R{t)-\lyi 



{x^C\t)R{t)-^C{t)xJs 



L ./0 



where 






x{t) 



(l + e|x-(t) |2)i/2- 
There exists a subsequence, denoted by e, such that 

almost surely (a.s.) 77“ (t) ^ V^{t) Vt. 



(43) 



(44) 



(45) 



It follows that 

d\a{t)\‘^ = 4|a(t) l^a'(f) [A{t)a{t)dt + dw] + 2 [a{t)‘^ trQ (t) + 2a'Q{t)af\ <if(46) 




Optimal Controller Switching for Stochastic Systems 351 



and 

d? 7 “(f)|a(f)|^ = 4?y“(f)|a(f)pa'(f) [A{t)a{t)dt + dw] + r]^{t)\a{t)\'^x[C' R~^dy 
+ 2‘rf^{t) \a{t)‘^trQ{t) + 2a Q{t)a\ dt Vf G [0,tAr) / tj. (47) 

Upon integrating between (0,t), using the smoothing property of integrals and 
taking the expectation it follows that 

£ril^{t)\a{t)\‘^ = + 4£ I t]]^{s)\a{s)\^a' {s)A{t)a{t)dt 

Jo 

+ 2£ j rf^{s) [\a{s)\‘^trQ + 2a'Qaj ds. (48) 

Jo 

Using the fact that £rf^[s) = 1 Vs it can be shown that 

£r)'J{t)\a{t)\^ e £\^f + C 2 (^1 + £ 71^{s)\a{s)fds'^ (49) 

where C 2 is a constant independent of e and u{^. Using the Gronwall Lemma 
one immediately obtains 

£y^{t)\a{t)\^ e Cl. (50) 

Hence taking the limit as e ^ 0, the claim of the lemma immediately follows. 



Lemma 2. For any admissible control, u G X , and for all trajectories the cost 
denoted by J*(u(€)) is finite. 



Proof. It follows from the Gronwall Lemma that 



xi(t)| G C3 




(51) 



Using the definition of (36) and from Lemma (1) it follows that on the finite 
interval [0, tAf) there exists a constant Ci such that 

£v:{t)\a{t)\^ e Cl. (52) 

substituting into J*{u{^) the claim of the lemma is established. 



Lemma 3. r?"(t) is a JT* martingale. 

Proof. From Lemma 2.4.2 (pg 40) of [6] all that needs to be established is that 



£ 




7]"^ {t)\u{t)\‘^ dt < 00 . 



(53) 



Using the fact that the system is piecewise linear and rj'^it) is finite for all 
t G [0, fAr) it follows trivially that there exists a constant C3 such that 



£ 



tjv 



0 



rj^{t)\u{^\^dt < C3. 



(54) 
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The following Lemma is also needed in the proof of Theorem (2). 

Lemma 4. Suppose that the solution P{^ to the Ricatti differential equation 
(21) with initial condition P(0) = Pq is defined and positive definite on the 
interval [0,tAf) then 



min 6^ (x (tj\r)X fx{tf^) + f x [t)N{t)x{t) + u'{t)M{t)u{t)dt 
= 1,...,N \ Jq 

= min ( X (t]\[)X fx{t]s[) + [ x [t)N{t)x{t) + u'{t)M{t)u{t)dt 

1=1,. ..,N \ Jq 

(tAr)X/e(t 7 v) + j e [t)N {t)e{t)dt 



(55) 



Proof. Rearranging (25) we have 



£■“ £ 



£^ l^£ 



/ f 

X {tN)Xfx{t]\i) + X {t)N{t)x{t) + u {t)M{t)u{t)dt I Z‘ 

Jo 

(x(tAr) ® x(tjv)) Xf{x[tN) ® x(tAr)) + (x(fjv) ® x{tj\/)) Xfx{tN) \ 



X {tj\[)Xf{x{tN) ® x{NT)) + x [tN)X fift^) I Z^ 

ptN ^ 

/ (x{t) + x{t)) N{t){x{t) Z x{t)) + 

lo 

[x{t) Z x{t)) N[t)x{t) Zx (t)N{x{t) Z x(t)) I Z^ 

ftN 

£" l£ 



+ 



X {t)N{t)x{t) + u' [t)M{t)u{t)dt I Z* 



Uo 



(56) 



Using the orthogonality property of the Kalman filter [16], nsunely £ [(x Z x) Z x 
= 0 where Z is any matrix, it immediately follows that 



£U 

^£U I £U 



X (tjv)Xfx(tjv) 



X (tM)Xfx(tjv) 



ptN 



X {t)N{t)x{t) + u' {t)M {t)u{t)dt I Z* 



/■tN 



X (t)N{t)x{t) + u [t)M{t)u{t)dt I Z* 



ptN 



+ 8'^ \e {lN)Xfe(tN) + / e {t)Ne{t)dt . 



( 57 ) 



The result now follows immediately. 
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Abstract. In this paper, the problem of robust control of hybrid systems 
is investigated. The hybrid systems under study are characterized as 
systems whose inputs contain both analog variables and discrete actions 
and whose behavior belongs to an uncertainty set which includes both 
metric modeling errors and discrete structural uncertainties. 

For a generic class of uncertain nonlinear hybrid systems, a design me- 
thodology for hybrid state feedback which maps hybrid states to hybrid 
control variables is developed based on system performance. Robust sta- 
bility and performance of the closed-loop hybrid systems are established 
for systems subject to both analog modeling errors and structural un- 
certainties. 



1 Introduction 

In this paper, the problem of robust control of hybrid systems is investigat- 
ed. The hybrid systems under study are characterized as systems whose inputs 
contain both analog variables and discrete actions and whose behavior belongs 
to an uncertainty set which includes both metric modeling errors and discrete 
structural uncertainties. 

Most practical systems are hybrid in nature. These systems demonstrate 
the behavior of an anolog dynamic system untill they encounter certain abrupt 
structural or operating condition changes. Such changes can be triggered by un- 
expected causes such as component failures, or system structure variations such 
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as adding or removing of communication channels or controllers. If the transition 
of these operating conditions is sufficiently fast as compared to system analog dy- 
namics, one may model these changes as instantaneous discrete switching actions 
of system’s behavior. Associated with such discrete actions, the system changes 
its initial condition, dynamics, input/output variables, equilibrium points, as 
well as performance specifications. 

It is now well recognized that there are many types of hybrid systems for 
which a completely unified framework may offer nothing more than an abstract 
language. For hybrid systems whose components are dominantly discrete events, 
representation theory, supervisory control, computer simulation and verification 
become main issues. Many of the papers in [15] [16] [17] are along this line of 
research. On the other hand, from the view point of classical feedback control, 
hybrid systems may be regarded as a case of switching control of nonlinear sys- 
tems in which switching actions are taken alongside traditional analog feedback 
loops. Some results in this area are found in [6] [7] [10] [11]. In a generic sense, 
the work reported in this paper follows the main ideas of the second view point. 
More specifically, however, we view hybrid control as a special form of adaptation 
and focus on active hybrid control in which switching is a critical component of 
feedback design. In addition to classical analog robust methods, hybrid control 
is capable of adapting to large modeling errors and structural changes by tuning 
its control structures, on top of analog control loops. 

Conceptually, hybrid control systems can potentially achieve better perfor- 
mance than non-switching robust controllers, due to their ability of reconfiguring 
and reorganizing their control structures. However, the realization of these ben- 
efits depends on a delicate coordination of discrete and analog control variables. 
It is well known that uncoordinated switching may cause instability for the hy- 
brid systems, let alone performance. Some essential questions arise: How can 
switching decisions be made which will guarantee stability and improved perfor- 
mance of hybrid systems? Can hybrid systems be designed to offer robustness 
not only for traditional analog disturbances and metric modeling errors, but also 
structural uncertainties? How much switching (i.e., adaptation) is needed? 

Due to complications involved in designing hybrid control systems, a brief re- 
view of existing methodologies is in order. The dwelling-time switching strategies 
[6] [7], in which switching is purposely slowed down to allow sufficient dwelling 
time for an anolog controller to bring analog states close to equilibrium points, 
can guarantee stability. An obvious drawback is that it will lose performance by 
delaying switching decisions. The state space partition approaches [15] [16] [17], 
which first divide the whole state space into a finite set of regions and then de- 
sign hybrid systems in the discrete domain, can reduce hybrid systems to finite 
atomata. As a result, discrete-event theory can be applied. It, however, creates 
a daunting problem of complexity explosion and makes it difficult to address the 
issues of performance and robustness in the design procedures. 
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Recently, there are substantial research efforts in employing Lyapunov theory 
to analyze stability of hybrid systems [18] [19]. Since hybrid systems are special 
cases of nonlinear systems, this approach is certainly one of the most natural 
methodologies to apply. A key issue to be clarified seems to be: What are the 
essential and unique issues rising from hybrid systems? From this point of view, 
the recent work of He and Lemmon [18] is of substantial interest, which employs 
some intrinsic features from discrete-event part of hybrid systems to reduce the 
complexity of Lyapunov analysis. Since Lyapunov approaches are usually very 
conservative, even for linear systems, it is of essential importance to address the 
issues of performance in addition to stability. 

The approach we propose comes from the following observations which stem 
from some of our work on practical hybrid systems: 

1. While stability is of important concern in hybrid systems, design of active hy- 
brid systems must demonstrate improved performance and robustness. Many 
practical hybrid systems are either stable or can be easily stabilized without 
using switching. More often than not, hybrid mechanism is introduced to 
improve performance and robustness, rather than nominal stability. 

2. Switching decisions should be made based on potential improvement on per- 
formance. As a result, active switching decisions must be pursued which do 
not slow down switching decisions on purpose. 

3. Switching decisions should be made based on the best utility of analog robust 
control methods. Hence, switching is not needed unless system performance 
cannot be achieved by robust analog control. This conforms to the idea of 
information-based adaptation, introduced by Zames [14]. 

In this paper, a design methodology for hybrid state feedback is introduced. 
The method employs traditional robust analog control for the analog state feed- 
back, and a performance-guided switching to form a discrete state feedback. For 
a generic class of uncertain nonlinear hybrid systems, robust stability and per- 
formance of the closed-loop hybrid systems are established in the presence of 
both analog modeling errors and structural uncertainties. 

The main idea of this paper stems from some simple features of classical 
optimal and robust control systems: 

1. LQ Control: For linear time- invariant systems, optimally designed controllers 
via Riccati equations always guarantee stability. Hence, additional testing 
of Lyapunov stability is unnecessary. This fact can be extented to include 
robustness when LQ performance indices are expanded to include additional 
terms to compensate potential modelling errors. 

2. H°° Control: Solutions to the HJI inequalities, which give sufficient condi- 
tions for H°° performance bounds, always guarantee input /output stability 
and Lyapunov stability. This is also true when modeling errors arc intro- 
duced. 




Robust Control of Hybrid Systems; Performance Guided Strategies 359 



These basic understandings suggest that if system performance indices are ap- 
propriately selected, optimality of performance, or mere boundedness of perfor- 
mance, can provide stability and robustness. Potentially, this idea can be used 
to overcome the conservativeness of Lyapunov analysis or Lyapunov design. 

The rest of the paper is organized as follows. The main structure of hybrid 
systems and control strategies considered in this paper is defined in Section 
2. Section 3 is devoted to a more detailed formulation of problems studied in 
this paper. The main technical results are the performance-guided hybrid con- 
trol strategies presented in Section 4. A techinical condition, called performance 
dominant conditions, is shown to be essential for applying the strategies. Robust 
stability and performance of the strategies are rigorously established. The effects 
of using switching penalty on hybrid robustness are discussed in Section 5. Sim- 
ulation results are presented alongside theoretical development to illustrate the 
main ideas and performance of the closed-loop hybrid systems. Finally, some 
conclusions are drawn in Section 6. 

Some of the main ideas of this paper have been employed in practical auto- 
motive control problems [18]. While these applications indicate the usefulness of 
the methodology introduced in this paper, they reveal also that there are many 
open issues to be resolved to narrow the gap between theoretical results and 
practical constraints. 

2 Hybrid Control Systems 

2.1 Hybrid Systems 

Hybrid systems studied in this paper, as depicted as in Figure 1, are input/output 
systems whose inputs contain both analog variables and discrete actions. Analog 
variables take real values in their respective intervals, and discrete actions can 
assume only a finite number of values. Consequently, one may label discrete 
actions with a finite set of symbols. 

In the absence of discrete actions, i.c., when the system is in a hxed discrete 
state s, a hybrid system resembles an anolog input/output system. Hence, it- 
s evolution behavior can be reasonably modelled by traditional mathematical 
models for dynamic systems, such as differential or difference equations, transfer 
functions, etc. 

When a discrete action is activated, the evolution of the system is inter- 
rupted, and its behavior undergoes an abrupt change. The change may entail 
a switching in system structure, a jump in internal states, a reconfiguration of 
inputs and outputs, a shift of equilibrium points, as well as installation of new 
design specifications. This abrupt change is summarized by saying that the sys- 
tem is switched from the old discrete state s to a new one Si. Consequently, we 
can represent discrete actions by cr = (s, s’l). 
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Traditionally, analog inputs are decomposed into (uncontrolled) disturbances 
d and control variables u. Similarly, discrete actions can be divided into (uncon- 
trolled) discrete uncertainty 5 and controlled discrete actions a. Further, outputs 
will be grouped into measured outputs j/m and unmeasured ones When nec- 
essary, we will use a to denote both S and a; and use y to denote both and 

Vu- 



d 



u 



a 



Hybrid Systems 



Vu 



Vm 



S 



6 



Fig. 1. Hybrid Systems 



Mathematically, a hybrid system is a mapping 

(s.y) = H(u, a;d, 5). 

The mapping is not completely known. Generically, the information about H 
is given by an uncertainty set f2 which contains H. 17 will be specified in the 
subsequent sections. 

2.2 Models 

At a given discrete state s, the system output is related to the analog inputs u 
and d via an uncertain mapping 

y = Gs{u,d) 



where the information about is given by the uncertainty set f2g which contains 

G, 
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A discrete (either controlled or uncontrolled) action cr = (s, si) at t causes 
the switching of the system from state s to state si 

y = Gs^{u,d) 

with a new initial value y(t+) which may be different from y{t-). Observe that in 
addition to the observed jumps from y{t-) to y(t+), the switching may also result 
in an unmeasured internal disturbance. Here, the effect of switching disturbance 
on the system is lumped as part of d. Of course, in the special case where the 
entire analog state x is measured, i.e., y — x, such jumps will be all included in 
changes from y{t-) to y(t+). 

Evolution of the discrete state s can be modelled as a discrete-event system 
in which discrete modes are represented by discrete states and discrete actions as 
discrete events. Ti'ansition and evolution of discrete states can then be modelled 
by an automaton. 

2.3 Hybrid Control 

Hybrid control studied in this paper is limited to causal mappings from measured 
output ym and discrete state s to controlled analog variable u and discrete action 
a 

{u,a) = d>{ym,s) 

such that the resultant closed-loop system 

is,y) = U{^{ym,s)-,d,5) 

demonstrates desired behavior in the presence of d, 5 and uncertainty Q. 

Generally speaking, design of such hybrid systems includes the following 
components. 

Analog Feedback: Robust Stability and Performance Analog feedback is 
responsible for determing analog control strategies u — 'Ps{ym) at a given 
discrete state s for the system y — Gg{u, d) so that robustness, stability and 
performance of the analog closed-loop subsystem y — Gg(Fg{ym), d) can 
be achieved in the presence of model uncertainty and disturbances. These 
are conventional systems described by differential equations, difference equa- 
tions, frequency responses, etc., and controlled by using robust, optimal or 
adaptive control methodologies. It will be shown that the essential infor- 
mation which must be communicated to discrete feedback is the achievable 
robust performance for the subsystem. 

DES Supervisors: Enabled Event Sets At a given discrete state s, DES su- 
pervisors determine which events are to be disabled. As a result, the output 
of the supervisors is a set Sdes{s) containing all the events deemed admis- 
sible by the supervisors. This set must be available to the discrete feedback 
for making switching decisions. 
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Discrete Feedback: Constrained Prioritation Discrete feedback is respon- 
sible for making switching decisions. More specifically, at any given time the 
discrete feedback must determine controlled discrete actions based on infor- 
mation submitted from hybrid states, DES and analog systems. Symbolically, 
the discrete state feedback is simply a constrained prioritation: 

inf 'P(o’) 

rr€Sc(s) 

where a is the discrete event which is to be selected from the constrained set 
ii'c(s) at the current discrete state s. The function V{a) is a priority function 
which assigns priority orders to different event commands a. Note that V is 
not necessarily numerical. It only requires a relation G . For instance, a\ G ct 2 
will simply mean that in the evaluation process, the event cri is judged as 
proceeding <72, and hence is a better choice. As a result, the main task for 
establishing the discrete feedback is to construct the constrained event set 
I7c(s) and priority function V . 

1. Constrained Event Sets 

Usually, for a given discrete state s, the constrained set Uc(s) can be 
constructed from certain conditions. Suppose at the state s, the uncon- 
strained set of all possible events is given by 

S{s) = {e, CTi = (s, si),^cr„ = (s,s„)} 

where e is the null event (i.e., no discrete switching occurs). Obviorsly, 
Uc(s) G U(s). 

Examples of possible constraints which reduce U(s) to Uc(s) are nu- 
merous. We will list some typical cases one often encounters in practical 
problems. 

— Discrete Constraints. This is provided by DES supervisors. At the 
current discrete state s, the DES supervisors provide a set Sdbs{s) 
of enabled events. Consequently, Uc(,s) G A'£)£;s(,s). 

— Analog Constraints. For example, at time t, analog constraints may 
be expressed as the conditions: “If the measured output ymij) be- 
longs to a set F, then <Ji — (s, si) is disabled.” 

— Hybrid Constraints. Such constraints will depend on both discrete 
and analog states. Take, for example, the hypothetic condition in an 
automotive control problem: “if gear(t®T) = 1 and gear{t) — 2 and 
vehicle speed v{t) > 20, then the shift (2, 1) is not allowed,” where T 
is a small time interval. In practice, this constraint will prevent gear 
chattering between Gear 1 and Gear 2. Apparently, this constraint 
mandates the memory of the discrete state s. Another possible case is 
given by the constraint “if gear{t) — 1 and u G 40 or u G c > 0, then 
the noiiswitching decision e — (1,1) is not allowed.” This reflects 
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the fact that staying at Gear 1 is not desired unless the vehicle is 
cruising at low speeds. 

The issue of constructing T'c(s) is currently under investigation and will 
not be covered in this paper. 

2. Priority Functions 

The constrained set T'o(s) provides the freedom for the discrete feed- 
back to make switching decisions at state s. The primary responsibility 
of the discrete feedback is then to make sound decisions within the set. 
This responsibility can be formulated by using the priority function V, 
which rank orders the events in T'c(s) on the basis of stability, system 
performance, robustness, coordination with other systems, environmen- 
t conditions, etc. When the task can be defined and evaluated off-line, 
the priority function can be constructed a priori, and implementation of 
the discrete feedback becomes a “simple” finite optimization procedure. 
In the more complicated, and more realistic, situations where priori- 
ty functions rely on environment conditions, priority functions must be 
constructed on-line, and the design of the discrete feedback can become 
much more involved. 

In this paper, we will focus on the design of discrete feedback. We will demon- 
strate that when system performance measures satisfy certain dominant condi- 
tions, they become viable candidates as priority functions in designing discrete 
feedback. 

3 Problem Formulation 

While practical systems seldom allow full state measurements and state feedback, 
it is beneficial for understanding achievable performance and design limitations 
to study first the “simpler” cases of state feedback. This can be viewed as a 
special case of our general formulation by identifying the measured output t/m 
with the state x, Pm = x. Technical results of this paper are limited to design of 
hybrid state feedback. 

In this and subsequent sections we will need the following basic notation. 
]R,(D, Z denote the real numbers, complex numbers and integers, respectively. 
The absolute value of x G (C is \x\, and the real part of x is 3?(x). 

For a vector x G R", its Euclidean norm will be denoted by ||x||. For a matrix 
M G R"^'", its largest and smallest singular values are denoted by a{M) and 
a{M), respectively. 

denotes the spaces of sequences G R",t G Z}, for which 

||u||oo = sup ||u(t)|| < oo. 
tez 

For a time t G R, and t+ will denote its left and right limits, respectively. 
Hence, x(t_) is the left limit of x at t. 
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3.1 Hybrid Plants In State Space with Uncertainties 

In this paper, we are considering hybrid systems expressed in the state-space 
form 



X = f{x, u, d; s; t) + h\(x, u, d\ s; t), 
s = DES{S,a) (1) 

where x{t) E R" is the analog state, u{t) G the analog control input, d E fid 
the analog disturbances; s G EJ is the discrete state, 5 G is the uncontrolled 
discrete action, fig is called discrete uncertainty set. a{t) E S is the discrete 
control action, and if is a finite set of admissible discrete control actions. A E fi 
represents model uncertainties, f? is called model uncertainty set. f is assumed 
to be continuous in x and u. DES is a discrete event automaton. 

The discrete action a (i.e., d or a) takes the form of a = (s, si) where s 
is the old discrete state and si the new one. Associated with a discrete action 
at t, the system nominal dynamics will switch from x = f{x,u,d;s;t) to i = 
f{x, u, d; si; t), and the state x will jump from x(f_) to x(t+). 

It is well known that switching among stable systems may result in instability 
if switching is not carefully coordinated, as demonstrated in the following simple 
example. 

Example 1 Consider two linear systems 

X = AgX + BgU, s — 1,2. 



Suppose that for a given discrete state s, a state feedback Kg is designed such 
that the corresponding closed-loop system 

X = (Ag ® BgKs)x 



is asymptotically stable. It appears that for this system, the discrete event system 
should have only two states: s = 1 and s = 2, and both states are stable ones. 
However, it is easy to construct examples in which there exist (infinitely) many 
switching squences which will result in instability, even though both subsystems 
are stable individually. 

For example, suppose 



Ai (g) BiKi 



®100 20 
200 0100 ’ 



A2 0 B2K2 



0100 200 
20 0100 



The two closed-loop subsystems are stable individually. However, when switching 
between the two systems occurs at kT, k — 1,2, EEE, with T — 0.01, and x(0) = 
[1 0]^, the resultant hybrid system is unstable, as shown in Figure 2. 



This example suggests that analog control and discrete actions must be co- 
ordinated to achieve ultimate goals of robust stability and perfornianec. 
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Fig. 2. An Unstable Hybrid System 



3.2 Robust Hybrid Control Problems 

Suppose a hybrid system is modeled by (1). Without loss of generality, we assume 

/(0,0,0;s;t) = 0, Vs G R, 

namely, x = 0,u = 0 is an equilibrium point of the nominal system. 

At the current time t, the performance index for expected future performance 
is expressed generically as 

/ OO 

w{t 0 t)h{x{T),u{T),T)dT (2) 

where h{x,u,t) G 0 is a penalty function on analog state and control variables. 
The moving-window weighting function w{t) represents a tradeoff between tran- 
sient and persistent performance. More often than not, active switching is used 
in practical systems to achieve better transient performance (e.g., gear shift- 
ing to meet an acceleration command and better fuel economy). As a result, 
w{t), t > 0, is often a monotone decreasing function of time, although this is not 
an essential assumption for theoretical developemnt. 

Penalty on switching will be introduced later in the design process. Switching 
penelty does not enter the system performance index directly. But rather, it will 
become cither a constraint or an additional term in switching decisions to achieve 
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tradeoff among system performance, switching frequency, robustness, etc. While 
it has been well established in classical control problems that performance indices 
shall penalize deviations of x and u from 0 (the equilibrium point under study) , 
a reasonable penalty on the discrete action a is often its switching rather than 
its discrete states before or after the switching. For instance, to avoid busy 
shifting of transmission gears, the shifting, rather than gear positions, should be 
penalized. 

Definition 1 A hybrid state feedback consists of two mappings which relate the 
discrete actions and analog control to analog and discrete states: 

a = (s, si) = s), u{t)^g{x,s), 

where g{0, s) = 0 for all s G IJ. ^ and g will be termed as discrete state feedback 
and analog state feedback, respectively. 

After applying a designed hybrid state feedback, the closed-loop hybrid sys- 
tem can be expressed as 

i = f{x, g{x, s),d; s; t) + A(x, g{x, s),d; s; t), 
s = DES{S,4>{x,s)) (3) 

The assumption < 7 ( 0 , s) = 0 implies that the equilibrium point of the nominal 
closed-loop system under study is a: = 0. 

Problem For the system (1) and a given performance level g, design a discrete 
state feedback (f and an analog state feedback g such that for all A £ d € 
and S G Uj the equilibrium point x = 0 of the closed-loop system (3) is stable (in 
the sense to be defined later) and the system achieves the uniform performance 
level g 

sup Jt{x,g{x,s), (fix, s)) G g. 

t 



4 Analysis of Hybrid Systems 

Usually, practical hybrid systems employ active switching to improve system per- 
formance. This is exemplified by automotive transmissions, VDE engines, swirl 
control, hybrid vehicles, etc. Consequently, one may be tempted to use direct- 
ly the actual practical performance measures, say fuel economy, to determine 
switching actions. A potential danger of such strategies is that switching based 
on these performance measures may actually lead to a loss of stability. A central 
question is: What type of performance measures implies stability? Performance 
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dominant conditions introduced below define a class of performance measures 
which imply stability. 

It should be emphasized here that the notion of stability we employ in this 
paper is not Lyapunov stability. Consequently, the results presented in this paper 
should not be confused with those from multiple Lyapunov approaches. One 
exception is the case of LQ hybrid design for which Lyapunov stability can also 
be established. 

4.1 Performance Dominance Conditions 

In the subsequent subsections we will introduce some performance guided hybrid 
control strategies. The strategies rely on the fact that if performance measures 
have certain dominance features, uniform boundedness of them will guarantee 
the uniform boundedness of analog states. As a result, coordination of hybrid 
control variables can achieve transient performance and boundedness of signals 
(stability) simultaneously, without explicitly limiting switching rates to guar- 
antee stability. This will allow active switching actions, as often required by 
practical hybrid control problems. Hence, we will first introduce a key lemma 
which explains conditions under which performance measures become dominant 
over stability. 

Consider an analog nonlinear time-varying system in state space form 

(4) 

where x{t) G R" is the state variable. To accommodate possible switching ac- 
tions in hybrid systems, f{x, t) is assumed to be only piece-wise continuous in 
t. Suppose a performance measure is imposed on the system 

/ OO 

h{x{r), T)dr 

where h{x, t) G 0. 

Definition 2 Performance Dominant Conditions 

The performance measure J is said to be dominant for the system (4) if there 
exist constants c,ki,k 2 G 0 such that for alH G 0 

\\f{x{t),t)\\ G kih{x{t),t) + c, when \\f{x{t),t)\\ G 1, (5) 

||x(t)|| G k 2 h{x{t),t), when ||x(t)|| G 1. (6) 

Essentially, these conditions stipulate that: 

1. h{x, t) dominates the rate x of state evolution. As a result, any finite escaping 
of states will certainly show up in the performance measure J. 
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2. Large persistent x values must be detected in J, avoiding the situation in 
which the tail of x remains large even when J is bounded. 

The main utility of performance dominance is eleborated in Lemma 1. 



Lemma 1 Suppose J is dominant for the system (4)- If 



then 



sup J(t) G IX < oc, 
t>o 



Halloo G (fci + k 2 )ii + 2 + c. 



( 7 ) 



Proof: Suppose at some time t G 0, ||a;(t)|| = M. We will show that M G 
T /c 2 ))U + 2 + c. 

For any r G [t,t+l], 



x(t) = .t(t) 0 / f{x{9),9)d6. 



It follows that 



\x{t)\\ G ||a:(T)|| + / Wf{x{9),9)\\d9 



e lla^WII + J Wf{3^{0),^)\\d9. 

Let = {0 G [t, £ + 1] : \\f{x{9), 0) || < 1} and 1?2 = [£, £ + 1] G l2i. Then, by (5) 



|a;(£)|| G ||x(t) 



d9 



I 



>02 



[kih{x{9), 9) + c]d9 



G ||a:(T)|| + 1 + c + fci J(£) 
G ||x(t)|| + 1 + c + kin 



by (6). Consequently, by taking dr on both sides, noting that ||x(r)|| G 
1 + k 2 h{x(T),T),\/x, we obtain 

ft+i 



(||a;(r)l| + 1 + c + kip)dT 
J [k 2 h{x{T) , t) + l]dT + I + c + kill 



G k2pi T 1 Y 1 Y c A /ci/i. 

Since the right hand side is independent of £, we have 

ll^^lloo G (^1 T k 2 )p + 2 + c 

as required. 



□ 
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The notion of stability given by (7), namely boundedness of state variables, 
is not a Lyapunov stability. This is more closely related to the notion of stability 
used in adaptive control. This is not a surprise, considering that switching control 
is a special form of adaptation. It should also be pointed out that additional 
conditions can be introduced if asymptotic stability of x is required. This is 
shown in Lemma 2. 

Example 2 It is noted that for linear time-varying systems x = A[t)x with 
supj>o d(7l(t)) G fc, all quadratic measures J{t) = x^Qxdt. with Q > 0, are 
dominant with c = 0. 

Indeed, for ||a:|| C 1, x"^Qx C 2 ^iQ)\\x\\'^ £ ||2:|| • That is. 

Furthermore, the condition ||yl(t)x'(t) || G 1 implies 1 G ||^(t)x-(t) || G fc||x'||, or 
||x'|| G p Hence, 

Consequently, 

k 

\\A{t)x{t)\\ G fc||x|| G „ Qx. 

3L [Q) 

It follows that 

x^Qx G S.^{Q)\\x\\'^ G ~ ||x||. 

k 

4.2 Performance Dominance Conditions for Hybrid Systems 

Consider a hybrid system consisting of I nonlinear nominal subsystems 

x{t) = f{x{t),u{t);s;t), s=l,^l, (8) 

where / is continuous in x and u. 

Suppose for each subsystem, an analog bounded nonlinear state feedback 
u — g{x,s), s — 

with ||( 7 (x, s)|| G II X II, has been designed such that X = 0 of the s-th closed-loop 
subsystem 

X ^ f{x,g{x,s);s;t) (9) 

is asymptotically stable. Furthermore, at time t a performance measure 

/ OO 

w{t ® t)h{x, u, r)dT 
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is imposed for design of the hybrid system, where h{x, u,t) € 0 and the moving- 
window weighting function w{t) satisfies 0 < wi G w{t) G Wh,'it G 0. 

First, we will show that the weighting function w, which is often selected to 
reflect tradeoff between transient and persistent performance, will not affect the 
dominance of a performance index. 

Lemma 2 Suppose that in the special case of unweighted performance, i.e., w = 
1, J is dominant for (9) for all s = 1, 1. If a switching strategy results in a 

uniformly bounded performance 

sup J(t) G rj < 00 , 
t>o 

then the state x of the closed-loop hybrid system, is bounded by 

Ikiloo £ ^ V + 2 + C. (10) 

Wl 

If in addition there exists an integer rn > 0 and a constant k > 0 for which 

\\x{t)\\'^ G kh{x,u,t) (11) 

then x{t) 0, as t oo. 

It should be pointed out that Lemma 2 is slightly different from asymptotic 
Lyapunov stability which requires continuous dependence of ||x||cx) on ||x(0)||. 
The notion of stability here resembles those used in adaptive systems where 
asymptotic stability is interpreted as boundedness and convergence of state vari- 
ables. Unless specified otherwise, this notion of asymptotic stability will be em- 
ployed in this paper. 

It is noted in passing that for linear quadratic problems in Example 2, the 
conditions of Lemma 2 are satisfied with c = 0 and m = 2, since for all x, 
x^Qx G ct^(( 5)||2;|P when g_{Q) > 0. 

Proof: We first observe that after applying the analog feedback, we arrive at 
the closed-loop analog subsystem 

X = f{x,g{x,s)-s-t). 

Since the unweighted J is dominant, for some c,k\,k 2 G 0 

ki 

\\f(x,u;s-,t)\\ G kih(x, u,t) c G — w(r ^ t)h(x,u,t) c, if ||/(x, u; s; t) || G 1 

Wl 

fci 

|x(t)|| G k 2 h{x,u,t) G — w{t <S> t)h{x,u,t), if ||x(t)|| G 1. 




Robust Control of Hybrid Systems; Performance Guided Strategies 371 



As a result, the weighted performance J(t) is still dominant for the closed-loop 
systems. Now by Lemma 1, the uniform boundedness of J{t) implies that 



II II (ki k2] 

cxD G b + 2 -f c. 

Wl 

The conclusion x{i) ^ 0, as f ^ oc follows from the fact that 

/■oo roo 

/ ||x(t)||'"(it G / kh{x,g{x),t)dt 



Jo 



Wl Jo 



pOO 

/ w{t)h{x, g{x),t)dt 

Jo 



< oo. 



□ 

Lemma 2 reveals that under the performance dominant conditions, if a hybrid 
control strategy results in a uniformly bounded performance J, then stability 
or asymptotic stability of the closed-loop hybrid system is automatically guar- 
anteed, independent of the choice of weighting functions, provided that they 
are uniformly bounded away from 0. This will be important in hybrid control 
problems, where weighting functions must be tuned to reflect desired tradeoff be- 
tween transient and persistent performance. In other words, switching strategies 
which improve the performance measure J{t) will always imply stability. This 
will allow a design of hybrid state feedback without explicitly limiting switching 
rates. 

5 Performance Guided Hybrid Feedback 

Lemma 2 suggests a natural hybrid control strategy which employs J{t) as a 
guideline in the switching decision of discrete actions. 

5.1 General Results 

A hybrid control must specify both analog and discrete feedback mappings. In 
this paper, we will focus on the design of discrete state feedback, i.e., switching 
decisions. Hence, we will assume that for each subsystem, an analog robust state 
feedback has been designed. 

Performance Guided Hybrid Feedback: 

1. The Analog Feedback: Suppose that for each discrete state ,s G EJ. an analog 
bounded state feedback 



u{t) = g{x{t),s) 
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is applied such that x = 0 of the s-th closed-loop subsystem 
is asymptotically stable. 

2. The Discrete Feedback: Suppose that at t and analog state the con- 

strained discrete action set for s is T'c(s). Other than the singular case 
^c(s) = (be., no switching is allowed), the discrete feedback must decide 

whether a discrete action should be activated. Let 

/ OO 

w{t 0 t)h{x, g{x, s), T)dr, 

i.e., the expected future performance when the discrete state remains at s 
after t. Suppose that a discrete action a — (s, si) G Sc{s) will result in a state 
jump from x(t_) to x(t+). Then, at time t, the discrete action a — (s, si) is 
determined by 

(s,si) = (p{x{t-),s) = argmin{J(x(t+),si;t) : (s, Si) G i^c(s)}- (12) 



In other words, the priority function for designing discrete feedback is ex- 
pressed as: ai = (s, si) G T'c(s) proceeds «2 = (s,S 2 ) G ^c(s)i be., ai < 02 , 
if 

J{x{t+),Si,t) < J{x{t+),S2,t). 



Hence, ai is a better choice than 0:2 if switching to si provides a better expected 
performance than switching to S 2 . In the competing case 



J{x{t+),Si,t) = J{x{t+),S2,t), 

one may select any one for switching.^ 

Note that we do not explicitly limit the magnitude of the state jumps. The 
key consideration here is their effects on achievable performance rather than 
their metric distance. 

Define 

Vo = J{x{t-,so),so,t). 

r]o is the performance achieved by using the non-switching control strategy 
at the discrete state sq for all time after . 



Theorem 1 If the performance index J is dominant for each closed-loop sub- 
system 

i = f{x,g{x,s);s;t) 

and satisfies (11), then the performance guided hybrid feedback guarantees 

^ The competing case may be resolved by using other criteria which are transparent 
in our formulation. 




Robust Control of Hybrid Systems; Performance Guided Strategies 373 



1. ||x||oo bounded, and x{t) 0, as t oo. 

2. At any time t, the resultiny performance J{t) of the hybrid control strategy 
is better than or equal to non- switching control, namely, 

J{t) e r?o- (13) 

Proof: Suppose that the switching sequence is s. Then, the system equation 
along this trajectory is given by 

i' = f{x,9{x,7)-,s]t) fo{x,t). 

By definition, the performance-guided hybrid feedback implies that at any time 

t, 

J{t) = min J(x, s, t) G J(x, 's, t) = r]o 

S 

which implies (13). 

Now, the performance dominant conditions for all subsystem imply that 

||/o(x,t)|| e max||/(a:, s,t)|| 

G kh{x,t) + c, when ||/o(x, f)|| G 1 

for some fc > 0 and c G 0. Consequently, all conditions of Lemma 2 arc satisfied 
and the first conclusion follows. i — i 



5.2 Hybrid Linear Quadratic Control 

In the special case where the underlying analog systems are linear time invariant 
and performance measures are LQ types, explicit computations can be easily 
obtained. In fact the stronger results of asymptotic Lyapunov stability hold in 
this case. For this reason and practical importance of LQ problems, we provide 
detailed discussions on hybrid LQ design in this subsection. 

Consider a hybrid system which consists of I linear subsystems 

X — AgX + BgU, s — 1,2, I, 

where x G M", u G R™, and all pairs (Glg,i?s) are controllable. 

A quadratic performance index is given by 

/•OO 

•■^(^) = j [x'^ {t)Qx{t) + {T)Ru{T)]dr 

where Q and R are real, symmetric and positive dchnite. If there is no switching, 
it is well known that for the s-th subsystem the optimal control, which minimizes 
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J , can be obtained from the positive definite solutions Kg to the algebraic Riccati 
equation: 

KgAg +A^Kg + Q(^ KgBgR-^Bj Kg = 0 

and the corresponding optimal linear state feedback is 

u{t) = ®R~^BJ Kgx{t), ret. 

The optimal performace can also be easily computed as 

Js{t) = ^x'^{t)Kgx{t). 

Obviously, if no switching occurs, the s-th closed-loop subsystem 
X — {Ag ® BgR^^Bg Kg)x — AgX 

is asymptotically stable. Hence, all eigenvalues Aj(Ts) of have negative real 
part. Let 

CTg = max |3?(Aj(^s))|- 

It follows that there exists a constant > 0 such that for any initial state x(0) 

\\x{t)\\ G Cse-^^^\\x{Q)\\. (14) 

Denote 

cr = max <Ji > 0 

c = min Cg > 0. 

Then, 

||x(t)|| G Cse^‘^"*||x(0)||. 

Intuitively, this inequality means that x{t) cannot decrease to 0 extremely fast. 
As a result, if infj^^ x^(r)Qx(r)dT is small, max,-gp ||a:(T)|| cannot be too 
big. Hence, boundedness of J will imply boundedness of ||a:'||.^ 

Suppose a sequence of discrete actions occur at 

to < ti < t2 < 8EG 

Lemma 3 For any given switching sequence, if the corresponding performance 
index is bounded 

/ OO 

[x"^ {t)Qx{t) + u'^{T)Ru{T)]dT E T] < OO, Vt 

then the equilibrium point x — 0 of the hybrid system is globally asymptotically 
Lyapunov stable. 

^ One can easily construct a case of nonlinear systems in which extremely large values 
of x{t) can escape the detection of .7 when x{t) can decrease to 0 extremely fast. 
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Proof: First. J{t) € implies 



[t)Qx{t)dt E q. 



Since Q > 0, for a = g^{Q) 

rOO roo 

r]E x^ {t)Qx{t)dt E a x^ {t)x{t)dt. 

Jo Jo 



Hence, 



x'^ {t)x{t)dt E — := rji. 



(15) 



We will show by contradiction that 



||x|joo = sup ||x(t)|| e 



1 / 2?7iCT 



1 ® ’ 



Hence, suppose that for some t G 0, ||x(t)|| — M > ]:\j Then for 
T G [t, t + 1], by (14) we have 

|| a ;( r )|| 2 G c 2 e - 2 -(- 0||^(^),|2 



It follows that 



/ x^ (r)x(T)dT G I 



rt+l 



2a 



-[I® e 



- 2(71 



However, by (15), x^(r)x(r)(iT G 771 , i.e., 



2cr 



■[1 ® e G ? 7 i 



or 



M G 



1 / 2?7icr 



1 ® 



This contradicts the hypothesis. Therefore, we conclude that 



X Lx) G - 



1 / 2'qia 



1 (g) 



( 16 ) 



Since for LQ problems 771 is always bounded by 



77 K\\X 

771 = - G 



a a 
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for some constant k > 0, the bound (16) implies that 



1 / 2k(T 

c y a(l ® 



|x(0)|| 



and a; = 0 is Lyapunov stable. 

Finally, it is obvious that J{t) E q < oo, Vt implies x{t) ^ 0, as t ^ oc, sinee 

Q > 0. |—| 



Lemma 3 shows that any discrete switching sequences which produce uni- 
formly bounded performance J will automatically guarantee asymptotic Lya- 
punov stability. Since this result does not require a minimum residing or dwelling 
time on any subsystems, fast switching can be accommodated. In particular, the 
following performance guided hybrid control will guarantee a stable closed-loop 
hybrid system. 

Performance Guided Hybrid Feedback: 

1. The Analog Feedback: If the current discrete state is s, then 

u{t) = Ksx{t). 

2. The Discrete Feedback: Suppose that the constrained discrete action set at 
s is given by T’c(s). At time t, if the current analog state is x{t), then the 
discrete action a — (s,si) is determined by 

Si = SiXgmm{x'^{t)Ksj_x{t) : (s, si) G A’c(s)}. (17) 

Define 

qo — min{a:^(0)Arga:(0) : s = 1, ^E, 1}. 

qo is the best performance achievable by using LQ non-switching control strate- 
gies. 



Theorem 2 The performance guided hybrid feedback guarantees that 

1. X = 0 of the closed-loop hybrid system is globally asymptotically stable in the 
Lyapunov sense. 

2. The resultant performance J(0) of the hybrid control strategy is never worse 
than the best non-switching control, 



J(0) G qo- 



(18) 
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Proof: 

By Lemma 3, we only need to show that any switching sequence produced 
by the feedback mappings will guarantee that 

noo 

J = / [x^ (t)Qx{t) + (t)Ru{t)]dt & rjo < oo. 

Jo 

Suppose that the discrete state sequence is 

s = j-0, 1,^, to - 0. 

For any integer > 0, the truncated switching sequence is defined as 

SN = {s(0), s{ti), EE£, s(t7v)}. 

The corresponding performance index is given by 



rti 



Jn^ + 5E£d 



rti 






rtN ro 

/ ^ 

'tN~l JtN 

{■In 



Hn-i 






switching criterion (17), 




fti 




Jn ^ f + 


/ +X^ {tN)KgUN-i)X{tN) 


Jo 


JtN~l 


fti 




= / +^F 


/ +X^ {tN-l)Kg(t^_^)x{t_ 


Jo 


Hn 1 



Further iterations lead to 



Jn e x^( 0 )A: 3 (o)x( 0 ) = 770 , 



(19) 



noting that s(0) is selected according to (17). 

Since the right hand side of (19) is independent of N , we conclude that 



J(0) = lim Jjv C r?o 

N—^oo 



as stated. 



□ 



Remarks: In concrete examples, it can be expected that the actual J(0) can 
be much better than 770 , that is, switching is useful for improving performance. 
However, there are cases of hybrid systems in which non-switching control is in 
fact optimal. For instance, if there exists sq such that ®Kg G 0, s = 1, <3E£, I, 
then the non-switching control with s{t) G a'o, t G 0 is the optimal choice. 
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6 Switching Penalty and Hybrid Robustness 

A key property which was employed to establish Theorems 1 and 2 is the mono- 
tone decreasing (non-increasing) of the performance index, alluded in the design 
of discrete feedback. The property is only valid if the plant model is exactly 
known and all discrete actions are control variables. For a plant modelled by (1) 
with both discrete uncertainty and analog modeling errors, the discrete feedback 
cannot guarantee the monotonicity of J. Consequently, uniform boundedness of 
J{t) may not hold, and hence stability cannot be guaranteed. We are therefore 
seeking modified switching strategies, which will guarantee robust stability and 
performance, in the presence of analog modeling errors and discrete uncertain- 
ties. 



6.1 Systems with Analog Uncertainty 

Consider a hybrid system with analog modeling errors 

X = f{x,u; S’,t) + A{x,u; s;t), s = l,€SS,l (20) 

where Z\ e is a term of analog uncertainty at the discrete state s. While the 
actual form or value of A is unknown, the nominal system / and the uncertainty 
sets I7s, s — 1, I, are given a priori At time t with the analog state x(t), the 
performance index for the system is given by 

/ OO 

w{t 0 t)h{x, u, t)cIt. 

Suppose that for each discrete state s, an analog feedback has been designed, 
u{t) — g{x{t),s), which achieves robust stability for the s-th subsystem (20). 

For a given analog state x{t) at t and a discrete state sequence {s(ri) : t € 

T\ < T2 < the nominal trajectory x*{r), t € t is the one governed by the 
nominal hybrid system 



x*{r) = /(a:*(r),5(x*(T),s(T));s(r);r) 

with the initial condition x*{t) — x{t). The resultant nominal performance is 
given by 

/ OO 

w{t ®t)h{x*{T),g{x*{T),s{T)),T)dT. 

In the special case when no switching occurs after t, we denote for s = 1, StE, I, 




j;{x{t),t) 



w{t 0 t)h{x* (t), g{x* (t), s), t)cIt. 
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Modeling errors A will perturb the system trajectories from the nominal 
ones. When the analog feedback mappings g are designed robustly, the closed- 
loop subsystem 

i = f{x, g{x, s); s; t) + A{x, g{x, s); s; t) 

is asymptotically stable, and the corresponding performance value for the non- 
switching strategy is bounded 

/ OO 

w{t 0 t)h{x{T) , g{x{T) , s),t)cIt < oo 

for all A G It is observed that Js{x{t), t) cannot be determined a priori since 
it depends on Zi. As a result, it cannot be used as an index for design. 

On the other hand, the nominal performance index J* and the worst-case 
performance errors 

es{x{t),t) ^ sup \Js{x{t),t) ^ s = 

are available at t, and can be employed in the design process.^ 

Hence, we introduce the following construction of disctrete feedback map- 
pings. 

Performance Guided Hybrid Feedback with Switching Penalty: 



1. The Analog Feedback: If the current discrete state is s, then 

u{t) = g{x{t),s). 



2. The Discrete Feedback: Suppose at the discrete state is s with the con- 
strained discrete action set Ac(s). The priority function is defined as follows. 
At time the new discrete state si is determined by 

Si = argmin{J^(x(t);t) -I- SP(s, si) : (s, si) e Ac(s)} (21) 



where the switching penalty term SP(s,si) is defined by 



SP(s, si) 



0 , Si = s 

es(x(t+)) + esi(x(t+)) + £o , Si ^ s 



and £o > 0 is a constant. 

® It should be emphasized here that “availability” of es{x{t),t) is of information nature. 
Namely, the information is available for computing es{x,t). Actual computational 
complexity of es{x,t) will obviously depend on the nonlinear system, uncertainty 
sets, performance measures, etc. These computational issues are not addressed in 
this paper. 
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Intuitively, introduction of the switching penalty term will impose cautious- 
ness in switching discrete states. More precisely, a switching from the current 
discrete state s to the new si is selected only if the nominal performance level 
J*^ (x(t+), t+) is better than J*(x(t+), (x(t+), t+)0es(x(t+), t+), namely 

In other words, switching will be decided only if the worst-case performance for 
the system si is better than the best performance of the system s. The following 
theorem establishes the robustness of this strategy. 

Theorem 3 Suppose that the performance index J is dominant for each subsys- 
tem. The performance guided hybrid feedback with switching penalty guarantees 

1. Robust asymptotic stability of the closed-loop hybrid system. 

2. The resulting performance J of the hybrid control strategy is bounded by the 
robust worst-case performance of non-switching control strategies, namely, 

JD min[J*(x(0), 0) + eg(x(0), 0)]. 
selj 

Proof: The proof is similar to that for Lemma 2 and is omitted. i — i 



Example 3 Consider a hybrid system with six discrete states, 
X = AgX BgU, s = 1, 2, 3, 4, 5, 6 



where 



Ai = 
A2 = 
A3 = 
A4 = 
A5 = 
Ae = 



1 ®5 
9 

1 2 

2 6 

0.7 6 

0.9 ®5.88 

1 ®7.8 

2.05 ®5.56 



Bi = 
B2 = 



3 9.1 
1.9 1.2 

2 2.5 
2 2.7 



, Bs = 

, B4 = 
Ss = 

Be = 



1 

1.5 



0.5 

0.4 



1 

05 



2 

1.1 



The analog state feedback gains Kg are designed by solving the standard alge- 
braic Riccati equations with performance weighting matrices Q — ^ 1 | I and 
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R — 0.1. Switching penalties are defined by the matrix SP: 



SP 



0 11111 
10 1111 
110 111 
1110 11 
11110 1 
111110 



D d, 



where d is a scaling constant. Here, SP(s,si) is the penalty for switching from 
discrete state s to discrete state si. 

For d = 0.06 (heavy switching penalty) and d = 0.001 (light switching 
penalty), simnlation results are presented in Figures 3 and 4. The optimal non- 
switching performance is compared to the performance guided switching strate- 
gy. Apparently, in this case the switching strategy offers a much better perfor- 
mance. It is observed that increasing switching penalty will result in infrequent 
switching (good) and a loss of performance (bad), which seems to be an under- 
standable tradeoff. 




4 



3 

u> 

c 



§ 

M 

1 



Ql ^ ^ ^ 1 

0 0.5 1 1.5 2 

Time (Second) 




Optimal Nonswitching Vs. Hybrid 




Fig. 3. Hybrid Control With Switching Penalty: d = 0.06 
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Optimal Nonswitching Vs. Hybrid 





Fig. 4. Hybrid Control With Switching Penalty: d — 0.001 



6.2 Systems with Discrete Uncertainties 

Analysis of Robust Stability For stability analysis, one may lump both 5 
and a as a discrete action a. Consider an uncertain nonlinear hybrid system 

X — f{x,u\ s\t) -\- A{x,u\ s\t), s = l,2,&S^l, (22) 

where Z\ G Cg is the modeling uncertainty. 

Suppose that for each discrete state s, a (nonlinear) state feedback u = g{x, s) 
has been designed such that the closed-loop system 

X ^ f{x,g(x,s);s;t) + A{x,g{x,s)]s;t), s^l,2,e^l (23) 



is robustly Lyapunov stable. More precisely, we assume that there exist Lyapunov 
functions Vs{x) for the s-th subsystem which satisfies that for some Ug > 0, bg > 

0, Cg >0 

ag||x||^ G Vg{x) G &g||x|p 



and 



dVgjx) 

dt 



G ®Cg X 
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along the trajectories of the system (22) for all Zi £ l?^. There are now numer- 
ous robust design methods for nonlinear systems which will guarantee robust 
Lyapunov stability. The reader is referred to [5] [9] for more reference. 

Denote 

7 s = — , and 70 = max 7 s. 

CLq s — 

First, we establish a property of discrete sequences under which asymptotic 
stability of the hybrid system is guaranteed. The reader is referred to [18] [19] 
for related stability results. 

Lemma 4 Suppose a discrete state sequence {s(t), t G 0} has its switching 
actions occur at ti, i — 0,1, SSiwith to < ti < t 2 < GGG If for some 0 < /3 < 1, 

]|x((t,)+)||^e/3||x((i,-i)+)|P, (24) 

then the equilibrium point x — 0 of the hybrid system is robustly asymptotically 
stable m the Lyapunov sense. 



Proof: The hypothesis implies that after switching 

llx((t*)+)||^ e f3^\\x{{to)+)f, z = 1, 2, SS 

Hence, 



^0, i^ 00 . 

Furthermore, for t G (ti,<i+i) the discrete state is s = s((ti) + ) and 



\\x{t)f& -Vs{x{t)) 

Os 

G — Vs{x{{ti)-o)) since Vg is a Lyapunov function of the subsystem s 

dg 

dg 

^ 7o||x((t.)+)f 
^ 7o/S^*lk'((to) + )||^- 

Since 0 < < 1, 

\\x{t)f G 7o]|x((to)+)|P, Vt G [to, 00 ) 



and 

These imply that x 
Lyapunov sense. 



x{t) 0, as t 00 . 

0 of the hybrid system is asymptotically stable in the 



□ 
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As a result of Lemma 4, to analyze asymptotic stability of a hybrid control 
scheme one only needs to establish conditions on discrete sequences s{t^), i — 
0, 1, such that the inequality (24) is valid. 

For a selected 0 < 77 < 1, define 

^ _ (70 ® v) as 

-7 S — 

V Cs 

and 

T — max{Ts : s = 1, SS, /}. 

Theorem 4 Suppose that the switching will cause bounded state jumps, uni- 
formly for all switching actions. Namely, for some constant k 

\\x{t+)f e 7 t||x(t_)f , 

at switching time t. If (5 — Kg < 1 and the discrete state seqxience s{ti), i — 
0, 1 , ^Sisatisfies 

U^t,^i&T, 7 = 1 , 2 ,^ 

then the equilibrium point x = 0 of the hybrid system is robustly asymptotically 
stable in the Lyapunov sense. 

Proof: By Lemma 4 we only need to establish the condition (24) for the given 
sequence. In the interval (L,ti+i), the discrete state is s = s((ti)+). By the 
hypothesis on the s-th subsystem, we have 

Vs{x{{ti+i)^)) e Vs{x{{ti)+)) (g) Cs x'^{t)x{t)dt. (25) 

First, wc show that if at any t G (tj, t,+i) 

\\x{t)f e ^\\x{{u)+)f, 

70 

then the condition (24) is satisfied. 

Indeed, in this case for any t E [ti, L+i) we have 

l|a;((^i+i)-)f e — Vs(x((t,+i)_)) 
as 

e — Vs(x(t)) 

Og 

G ^||x(t)||2 
Cts 

e io—\\x{(ti)+)f 
70 

er,\\x{{U)+)f. 
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It follows that 

||a;((t,+i)+)f e «||x((t,+i)_)||2 G e p\\xiiu)+)f. 

Hence, we only need to consider the case where for all t G (t^, ti+i) 

||x(t)f > —\\x{U)f. 

70 

It follows that 

\\xau+i)^)fi 



■V,{x{{U+i)^)) 

■[Vs{x{{ti)+)) (S>Cg I x'^ {t)x{t)df] by (25) 



G 7o||x((t,)+)ll"®- 



'u 

rtj+1 



x^ {t)x{t)dt 



Jti 



Therefore, 



e 7olk((ti)+)ll^ ® ® ti) 

as 7o 

= [70 ® —rj{U+i<»ti)]\\x{{U)+)f. 

da 



l^((^^+i)+)lP ^ ^[70 fK) —r]{ti^i ®U)]\\x{{U)-^)\\‘^. 



Clearly, the condition (24) will be satisfied if 



70 ® ® ti) G T], 



or equivalently 



{ti+i 0 ti) G 



70 0 ?7 

■ 

a. ' 



Therefore, the hypothesis (ti+i 0 ti) G T implies the condition (24). By 
Lemma 4, x = 0 of the hybrid system is asymptotically stable in the Lyapunov 
sense. 

Theorem 4 claims that if each subsystem is well designed to achieve ro- 
bust asymptotic stability individually, then the total hybrid system will remain 
asymptotically stable if switching causes only bounded state jumps and does 
not occur very frequently. This is the same idea as the dwelling time concept for 
logic switching in adaptation introduced by Morse [6]. 

Example 4 Using the same system as in Example 1, i.e., 



X = AgX^ s = 1,2 
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where 

_ r®100 20 "I ^ _ TOIOO 200 ' 

[ 200 ®100j ’ [ 20 ®100 ■ 

We can solve the Lyapunov equations 

AiPi + PiAj = A2P2 + P2AI = ®I 

to obtain 

_ ro.0068 0.0092] p _ [ 0-023 0.0092' 

“ [ 0.0092 O.O 233 J ’ “ [ 0.0092 0.0068 ■ 

Then 

b = a{Pi) = a{P 2 ) = 0.0274; a = a{Pi) = a{P 2 ) = 0.0028. 

Also, Q = I and c = (Q) = 1. It follows that 70 = 9.7857. 

Now, for a selected, say, /3 = 0.9, we have 

70 ® /3 a 

Jo = — 7 

p c 

9.7857 0 0.9 0.0028 

^ (L9 i 

= 0.0278. 

In this case, any switching sequences which cause state jumps bounded by 

\\x{t+)f < 

and occur at least To apart will guarantee asymptotic stability. 

Figure 5 demonstrates the hybrid system for switching at kT, k = 1,2, 

T = 0.04, starting at .t( 0) = [1 0]^, without state jumps at switching. 

Theorem 4 is useful in applications where infrequent switching is either phys- 
ically valid (component failures), acceptable (air conditioning switching, cruise 
control switching in automotive systems), or desirable (slow adaptation for slowly 
varying plants) . It is particularly useful in “passive” or “lazy” hybrid systems in 
which discrete actions arc not control variables. In such hybrid systems, one can 
employ the theorem for analyzing stability or constraining switching intervals. 

Discrete Robustness When a hybrid system contains uncontrolled discrete 
actions, control strategies must often be modified to accommodate uncertainties 
in discrete actions. Uncontrolled discrete actions can be further decomposed in- 
to two classes: Measured and unmeasured actions. Unmeasured discrete actions 
are those whose occurrence cannot be directly sensed or deduced uniquely from 
other measured quantities. Since these actions cannot be detected uniquely, con- 
trol strategies must be designed so that these discrete actions will not result in 
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Fig. 5. A Stable Hybrid System 



instability or total system failures. On the other hand, the measurable discrete 
actions can either be measured directly or deduced uniquely from measured vari- 
ables. For this type of uncontrolled actions, switching schemes can be devised 
so that adverse effects from these actions can be corrected, at least partially, by 
adjusting either analog mapping u or discrete action a. Typical examples of un- 
measured discrete actions include component failures in circuit boards, battery 
cell malfunctioning in electrical vehicle power supplies, etc. Measured actions 
are exemplified by cruise control switching which can happen any time but is 
known to the powertrain control system. 

Mathematically, we will consider a hybrid system modelled in state space 
form 



X = f{x, u\ s; t) 

S — DESi^d'f^^ ^rnt 

where 5u C is an unmeasured and uncontrolled discrete action; 5m C Om is 
a measured and uncontrolled discrete action; and a G if is the discrete control 
action. 

Suppose for each given measured discrete uncertainty 5m C a discrete 
feedback 



a = 4>{x, s) 
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and an analog state feedback 

u = g{x, s) 

have been designed such that the resulting closed-loop subsystem 

X = f{x,g{x,s);s-,t) 

s = DES{Su,Sjn,(f){x,s)) (26) 

is Lyapunov stable, in the sense of Section 5. The system (26) can be expressed 
concisely as 

X = f{x; s; t) 
s = DES{S,x) 

where 5 G 17^ is a generically labelled discrete action representing all possible 
combinations of 6u and 6m- 

By Lemma 4, to analyse asymptotic stability of the system (26) we may 
establish conditions on discrete state sequences s{ti), i — 0, 1, SSi such that 
the inequality (24) is valid. 

Define for some 0 < ?/ < 1, 

j- ^ ^ V) bs 

11 a.s 

and 

T — maxlTg : s G E)}. 

Theorem 5 If the uncontrolled switching 6 only causes bounded state jum,ps 

\\x{t+)f G K\\x{t^)f 

with l3 = KT] < 1 and the discrete uncertainty sequence 6{ti), t = 0, 1, QEEsatisfies 

ti®ti-i G T, i = 1,2,^ 

then the equilibrium point x = 0 of the hybrid system (26) is asymptotically 
stable in the Lyapunov sense. 

Proof: The proof is the same as that of Theorem 4. | — | 

Since 6 is uncontrolled, it cannot be actively selected. Therefore, Theorem 5 
is essentially a “passive” conclusion as Theorem 4. It claims that when analog 
and discrete feedback mappings are appropriately designed to achieve robust 
stability, the hybrid system will remain stable in the presence of discrete uncer- 
tainties, provided the uncertain discrete actions neither occur too frequently nor 
cause unbounded state jumps. 
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7 Conclusions 

The basic hybrid control structure and performance-guided hybrid control s- 
trategies introduced in this paper provide a framework in which stability, per- 
formance and robustness of hybrid systems can be analyzed and constructive 
design of hybrid feedback can be performed. This paper does not give any de- 
tailed treatment of constructing the constrained set of enabled discrete events, 
which is an important part of the total design procedure. This is currently under 
investigation. 
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Abstract. This paper presents reduction techniques from initialized 
rectangular automata with slope parameters to linear hybrid automa- 
ta. The translations are correctness preserving, thereby yielding semial- 
gorithms for the analysis of slope parameters. The automated analysis 
technique is used to synthesize new bounds on the independent clock 
drifts of processes in an audio control protocol. The emptiness problem 
for slope-parametric rectangular automata is proven undecidable. 



1 Introduction 

Most systems are described using design parameters — symbolic constants with 
unknown values. A major strength of linear hybrid automata is their ability 
to model such parameters within a framework amenable to automatic analysis. 
Model-checking algorithms for linear hybrid automata can be used to determine 
necessary and sufficient parametric constraints that guarantee that a safety prop- 
erty is met [CH78,HRP94]. However, the linear hybrid automaton model permits 
parameters to occur only in predicates relating to the variables’ values, not their 
flow fields. Thus slope parameters cannot be modeled directly. 

In this paper, we study systems where the parameters occur in the specifi- 
cation of the slopes of the variables. We are interested in questions of the form 
“how fast?” as opposed to “how far?” Slope-parametric rectangular automata 
are an extension of rectangular automata in that they permit the specification 
of parametrized bounding intervals on the slopes of the variables. Parametric 
analysis of the slopes involves hnding necessary and sufficient conditions on the 
slope parameters that guarantee that the automaton satisfies its safety require- 
ments. The automata we analyze are required to be initialized, i.e. whenever the 
flow field speciheation of a variable that is dependent on a slope parameter is 
changed, the variable is nondeterministically reset to an interval with constant 
bounds. We present semidecision procedures and .semialgorithms for the analy- 
sis of initialized slope-parametric rectangular automata. The automata are first 
translated into linear hybrid automata. The translation replaces parameters on 
the slopes of the variables with parameters on the values of the variables. Cor- 
rectness is preserved for all vectors of parameter values. Therefore, analysis of 
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initialized slope-parametric rectangular automata is reduced to analysis of linear 
hybrid automata, for which there are existing model checkers available. 

The technique is demonstrated on an audio control protocol. Using the sym- 
bolic model checker HyTech, we obtain new correctness conditions on the para- 
metric intervals bounding the independent drifting rates of the sender and re- 
ceiver clocks. 

We also prove the emptiness problem to be undecidable for initialized slope- 
parametric rectangular automata. Furthermore, the proof is easily modified to 
show undecidability of param,etric timed autom,ata with three clocks and one 
parameter, tightening the previously known undecidability result for six param- 
eters [AHV93]. 



Related work 

Boniol et al. study an analysis problem for a class of slope-parametric rectan- 
gular automata [BBRR97,BR97,RB98]. Their automata are uninitialized (those 
considered here must be initialized) and their slope specifications are affine in 
the slope parameters (here, they may only be the inverse of affine expressions 
in the inverses of the slope parameters). They require equality in their slope 
specifications, whereas here we allow intervals bounded by parametric expres- 
sions. They provide semialgorithms that use parametric-shaped polyhedra, for 
which specialized algorithms and implementations are required. In our approach, 
existing tools can be used for analysis. 

Bounds on the error rate of the audio control protocol are analyzed manually 
in [BPV94], and generated automatically in [HW95]. In those works, a common 
error parameter was used for the sender and the receiver clocks. The reduc- 
tion technique employed in [HW95] relied strongly on the fact that the drift in 
the clocks was the same for both the sender and the receiver: that method is 
insufficient for analyzing error rates that differ for the two clocks. 

Alur et al. give undecidability results for parametric timed automata, where 
all variables have rate 1 but their values can be compared to parameters [AHV93]. 
The undecidability result in this paper can be viewed as a companion to the un- 
decidability results in [HKPV95] for deciding emptiness of various extensions to 
initialized rectangular automata. 

2 Slope-parametric Hybrid Automata 

Linear hybrid automata combine the discrete dynamics of finite automata with a 
simple form of continuous dynamics. The continuous variables are subject to con- 
stant polyhedral differential inclusions [ACH+95]. This restriction corresponds 
to conjunctions of linear inequalities over the first derivatives of the variables. 
This model allows, for instance, the specification of drifting clocks via expres- 
sions of the form dx/dt € [5,7]. The restriction on the continuous dynamics 
simplifies computations over the continuous state space to manipulation of poly- 
hedra, enabling reasonably efficient automated analysis [AHH96]. We extend 
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the model to allow continuous dynamics that are expressed using symbolic con- 
stants, e.g. a variable may advance at a fixed unspecified rate fc > 0. We then 
wish to ask such questions as “for what values of k is the protocol correct?” 
Within each control mode, the specification of the rates of the variables can be 
given as a conjunction of linear inequalities over the set of the variable’s first 
derivatives and a set of slope parameters. However, such a model appears to be 
too general for efficient automated analysis. We instead focus on the subclass 
of slope-parametric rectangular automata, in which the flow fields are essentially 
restricted to rectangles. 

Let y be a set of variables and F a set of slope parameters. A convex linear 
predicate over y is a conjunction of inequalities over linear expressions in Y 
with rational coefficients, e.g. the predicate |j/i + 6^2 G ^ A y2 G 4 is a convex 
linear predicate over Y — {yi, j/2}- The set of convex linear predicates over Y is 
denoted C{Y). A parametric convex linear predicate over Y and A is a convex 
linear predicate over Y U F. The set of parametric convex linear predicates over 
y and F is denoted VC{Y,F). A convex linear predicate over the inverses ofY 
is a conjunction of inequalities over expressions that are linear (with rational 
coefficients) in terms of the form 1/y for y G y, e.g. the predicate ^ T ^ G | 
A ^ e 4 is a convex linear predicate over the inverses of y = {yi,y 2 }- The set 
of convex linear predicates over the inverses of Y is denoted CInviY). 



2.1 Syntax 

A slope-parametric linear hybrid automaton A is a system [X, F, V, flow, inv, init, 

final , E , jump) consisting of the following components: 

Variables: A finite set X = {xi, . . . , of variables. 

Slope parameters: A finite set F — {71 , . . . , of slope parameters. 

Control modes: A finite set V of control modes. 

Flow conditions: A function flow that maps every control mode to a flow 
condition in FC{X, F), where X — {ii, . . . , with Xj representing the first 
derivative of Xj with respect to time. 

Invariant conditions: A function inv that maps every control mode to an 
invariant condition of the form ipx A ipx, for ipx G C(X) and ip]< G CFnv{F). 

Initial conditions: A function init that maps every control mode to an initial 
condition of the form tpx /\^r, for ^px G £(A) and pr ^ CXnv{F). 

Final conditions: A function final that maps every control mode to a final 
condition in C{X). 

Control switches: A finite multiset E of control switches in V E V. For a 
control switch {v, v'), we say that v denotes the source mode and v' the target 
mode. 

Jump conditions: A function jump that maps every control switch to a jump 
condition in C{X U X'), where X' = {xj, . . . with x' representing the 

value of Xi after the control switch. 
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Fig. 1. Slope-parametric rectangular automaton A 



Figure 1 depicts a slope-parametric linear hybrid automaton with two contin- 
uous variables x and t, and two slope parameters 71 and 72 . There are three con- 
trol modes: vq, v±, and V2- The automaton can only start in a mode if the mode’s 
initial condition is satisfied. Within each control mode, the variables evolve ac- 
cording to the mode’s flow condition. Control of the automaton may remain in a 
mode only when the mode’s invariant condition is satisfied. In the graphical rep- 
resentation, we use guarded assignments for jump conditions, with the implicit 
understanding that any variable not reassigned retains its value after the control 
switch, e.g. the guarded assignment t = 2 Ax G 8 ^ x := 2 for the control switch 
(t>i, V 2 ) in Figure 1 represents the jump condition t = 2 Ax £ 8 Ax' = 2 At' = t. 
Control of the automaton may switch from one mode to another when there 
is a control switch for which the guard is enabled. The updated variables are 
simultaneously reassigned. Invariant conditions that are trivially true and initial 
conditions that are trivially false are not depicted graphically in the automata. 
Furthermore, we assume that whenever a convex linear predicate ipr over the 
inverses of F appears in the initial condition of a control mode v, it also appears 
as a conjunct in the invariant condition of every control mode v' for which there 
is a sequence of control modes connecting v to v' . The conjuncts (pr are omitted 
from the invariants in the figures to avoid clutter. In any case, interpreting the 
automata figures without the ipr in the invariants makes no difference as far as 
emptiness. 

Linear hybrid automata are simply slope-parametric linear hybrid automata 
for which the set of slope parameters is empty. In particular, slope-parametric 
linear hybrid automata admit the modeling of value parameters, which are vari- 
ables with global rate 0 that are never reassigned across control switches. 

Wc now dehne the subclass of slope-paramctric rectangular automata. A 
rectangular predicate over F is a conjunction of inequalities of the form y € a, 
where y CL Y , the relation G G{<,G,G,>}, and a G Q. The set of rectangular 
predicates over Y is denoted TZ{Y). A parametric rectangular predicate over Y 
and T is a conjunction of inequalities of the form y G a, where y G Y , the relation 
G G{<,G,G,>}, and a G Q U T. The set of parametric rectangular predicates 
over Y and F is denoted FTZfY, F). A slope-parametric rectangular automaton is 
a slope-parametric linear hybrid automaton such that every parametric convex 
linear predicate is a parametric rectangular predicate, and every convex linear 
predicate is a rectangular predicate. 







394 



II. Wong-Toi 



2.2 Semantics 

Transition systems. We provide semantics for slope-parametric linear hybrid 
automata as transition systems. A transition system T is a system 
Qt’ Qt)’ where Qt is the state set of T, the binary relation is the transition 
relation of T, the set specifies the initial states of T, and Q:^ is the set of 
final states of T. A trajectory of T is a finite sequence of states sqi si> ■ ■ • 7 Sfc such 
that So is initial, and for 0 G i < k, Si It is accepting if Sk is hnal. 

The transition system T is nonempty if there exists an accepting trajectory. 
A labeling for T consists of a set Lp of labels for T, and a labeling function 
At : Qt Lt that maps each state to a label in Lt- The transition system is 
nonempty for the label a G Lt under the labeling function At if there exists an 
accepting trajectory with hnal state Sfc for which \T{sk) — cr. 



Slope-parametric linear hybrid automata. A configuration of a slope- 
parametric linear hybrid automaton consists of a control mode, together with 
a point in For convenience, the point in is often written as (x, y) 

where x G K." represents the values of the variables, and y G K™ represents 
the values of the parameters. Let ip[Y a] denote the truth value obtained by 
evaluating the predicate (p with the value replacing each instance of the vari- 
able Pi in ip, for each i. The conhgurations of A are {(u,x, y) | inv{v)[X, L := 
X, y] is true}. A conhguration (u,x, y) is initial iff init{v)[X, F x, y] is true 
and final if final{v)[X x] is true. 

There are two kinds of transitions between configurations: jump transitions, 
which correspond to instantaneous control switches, and how transitions, which 
correspond to the variables continuously evolving while time elapses. For ev- 
ery control switch e G £1, we dehne the binary jump transition relation A by 
(u,x, y) A (u',x',y') iff e = {v,v'), y = y', and jump {e)[X, X' := x, x'] is true, 
i.e. the variables satisfy the enabling condition for the jump, all reassigned vari- 
ables have values consistent with the jump condition, and the parameters and 
other variables are unchanged. For every nonnegative real d G 0, we dehne the 
binary flow transition relation A by (v,x,y) A (ii',x',y') iff v = v' . y = y', 
and either (a) <5 = 0 and x = x' or (b) 5 > 0 and flow{v)[X, F (x' (g> x)/<5, y] 
is true, i.e. the control mode remains hxed and the rate at which the variables 
have changed over the S time period is consistent with the parametric how con- 
dition. For example, suppose X = {xi,X 2 \ and F — { 7 }. Let the how condition 
flow{v) for the control mode t> be Xi = 1 A ±2 = 2y. For > 0, there are how 
transitions between conhgurations of the form (v, (a, b), c) to conhgurations of 
the form (v, (a + S.fl+ 2cS ) , c) , since assigning xi the value ( (a + 15) (S> a)/i5, X 2 the 
value ((b + 2cS)^b)/b, and 7 the value c, makes the predicate /fow(u) equivalent 
to 1 = 1 A 2c = 2c which is equivalent to true. In such a transition — modeling 
the passage of 5 time units — the xi variable has progressed from value a to 
a + 5 at rate 1 and the X 2 variable has progressed from value 6 to 6 + 2c5 at 
rate 2 c, since the how condition spccihes the slope-parametric rate 2 q, and the 
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slope parameter 7 has value c. Observe that the values of the slope parameters 
remain unchanged in all transitions. 

Our semantics for a slope-parametric linear hybrid automaton A is the tran- 
sition system Ta — (Qa, -^a, Qa> Qa)j where Qa is the set of configurations of 

A, ^A is the binary transition relation IJeeA’ UdGK>o 

initial configurations of A, and is the set of final configurations of A. We 
often assume a standard labeling for Ta, consisting of the label set La being 
the set of real vectors M"*, and labeling function, denoted Aa, being the function 
mapping every configuration to its vector of slope parameter values. We say that 
A is empty precisely when Ta is. 

The final configurations of the automaton in Figure 1 are those for which 
the control mode is V2- It is not difficult to determine that the automaton is 
nonempty provided that 71 G 2 and 72 G 5 . 



Analysis problems 

We consider two problems for slope-parametric rectangular automata. The empti- 
ness problem for a slope-parametric rectangular automaton A asks whether A 
is empty. The parametric synthesis problem asks for the exact set of parameter 
vectors for which A is empty. 

3 Reductions to Linear Hybrid Automata 

We reduce our analysis problems over slope-parametric automata to analysis 
problems over linear hybrid automata. It is known that for linear hybrid automa- 
ta there is a semialgorithmic procedure for solving parametric noncmptincss, and 
that if the procedure terminates it yields the exact set of parameter vectors for 
which the automaton is empty [HRP 94 ] . We use bisimulation and mutual simi- 
larity to compare our original automata with our translated automata [ParSl]. 
Bisimilar and mutually similar automata are empty for matching sets of param- 
eter vectors, and thus our analysis problems over slope-parametric automata can 
be solved by analyzing the translated linear hybrid automata. 



3.1 Bisimulation and mutual similarity 

We define the notions of bisimulation and mutual similarity in our context. A 
state-labeled transition system is a pair (T, A'i') consisting of a transition system 
T and a labeling function Xq' for T. Let (Ti,Xq\) and {T2,Xt2) be two state- 
labeled transition systems with a common label set. Let Ti be {Qi, Q®, Q{) 
for i — 1 , 2 . The binary relation G G Qi G Q2 is a simulation of (Ti,Ati) by 
(T2,Xt2) iff the following four conditions hold: 

1 . For every state qi & Qi, there exists a state Q2 G Q2 such that qi & Q2- 

2 . For every state qi G Q{, for every state q2 G Q2, if qi G 52 then q2 G Qq- 
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3. For every pair of states q\ and q'^ in Qi such that (71 gj, if S (?2, then 
there exists a state such that q'-^ e q'2 and 52 -^2 92- 

4. If 91 e 92 then Ati( 9 i) = ^tA( 12 )- 

We say that (T2, Ats) simulates (Ti, Ati) if there exists a simulation of (Ti, Axi) 
by (T2, Axa)- The reverse of a relation G is defined by 92 G“^ 91 iff 91 G 92- 
A relation is a bisimulation between (Ti , A^j ) and (T2 , Axa ) if h is a simulation 
of {Ti,Xti) by (T2.AT2), and its reverse relation is a simulation of {T2,\t2) t>y 
(TijArj). Two state-labeled transition systems are bisimilar if there exists a 
bisimulation between them. Two state-labeled transition systems are mutually 
similar if they simulate each other. 

Proposition 1. If two state-labeled transitions systems are either mutually sim- 
ilar or bisimilar, then for every label, they are either both empty or both nonempty 
for that label. 

Let A and B be slope-parametric linear hybrid automata, and A^i and Xb 
labelings for their transition systems. We say that {A, Xa) simulates {resp. is mu- 
tually similar to, is bisimilar to) {B.Xb) if {Ta,Xa) simulates {resp. is mutually 
similar to, is bisimilar to) {Tb,Xb). 



3.2 Initialization 

There is a reinitialization condition, originally identified in [PV94], that is key 
to our results. We assume, for simplicity, that at every mode v, the parametric 
rectangular predicate over X and B for the flow condition can be written as 
a conjunction Aiiex where each l'^,u^ G Q U T, i.e. we assume 

finite nonstrict bounding intervals. A slope-parametric rectangular automaton 
is initialized if for every control switch e = (v.v'), and for every variable x, 
whenever both 

1. either If. xf If or uf fx uf (the flow specification for x changes), and 

2. {lf,uf,lf,uf}C]r 7^ 0 (the flow in either the source mode or the target 
mode depends on a parameter) 

are true, then x is reassigned into some constant interval by the jump condition 
of e, i.e. the variable x must be reinitialized along a control switch whenever the 
constraints on its flow are different in the source and target modes and the rate in 
either the source or target mode is defined using a slope parameter. Notice that 
this definition of initialization differs from that of [PV94,HKPV95], in that the 
variable x need not be reset every time its flow specification changes, only when 
cither the old or the new flow specification also depends on a slope parameter. 



3.3 Singular slopes 

A slope-parametric rectangular automaton is singular if for every control mode v, 
and for every variable x, fiow{v) is such that If = uf. Let A be an initialized 
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A 



Fig. 2. Initialized singular slope-parametric rectangular automaton A and a 
bisimilar linear hybrid automaton A 



singular slope-parametric rectangular automaton. We show how to construct a 
linear hybrid automaton A and a labeling such that (A, A^) is bisimilar to 
(A, Ayi). A variable x is slope-parametric if there exists a control mode for which 
the slope specihcation of x depends on a slope parameter, i.e. there exists a 
control mode v such that flow{v) implies x — ^( for some 7 £ F. The idea is to 
model every variable x with parametric rate 7 in A as a clock in A. The clock is 
reset whenever the variable x is updated. If the variable x has slope fc A 0 , then 
a configuration of A in which x has value d corresponds to a configuration in 
A in which the clock has value d/k. Predicates on x are scaled accordingly. An 
example of an initialized singular slope-parametric automaton A and its derived 
linear hybrid automaton A appear in Figure 2 . 

For simplicity, we assume for now that the invariant conditions of A imply 
that all slope parameters are strictly positive. Then A is derived from A by 
performing the following steps. 

1 . Replace each slope parameter 7j with a new value parameter pj, denoted 
[::p], which represents l/jj- Recall that a value parameter is a variable that 
has rate 0 in all control modes, and that is never reassigned by the jump 
conditions of the control switches. 

2 . Replace each convex linear predicate p over the inverses of the slope param- 
eters 7j with the convex linear predicate p over the value parameters 
obtained by substituting instances of 1/7^ in (p with [— ]. 

3 . For every slope-parametric variable x, and for every control mode v such that 
X has slope specification of the form i; = 7 in u for some slope parameter 
7 £ T, we perform the following four steps: 

(a) Replace the conjunct i; = 7 in the flow condition for v with the conjunct 
x = 1 . 
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(b) Replace every inequality of the form x G d, for a constant d £ Q, in the 
initial, invariant, and final conditions of u, with the inequality x G d[-]- 

(c) Replace every inequality of the form x & d, for a constant d £ Q, in 
the jump condition of every control switch of the form (v,v^), with the 
inequality x £ 

(d) Replace every inequality of the form x' £ d, for a constant d £ Q, in 
the jump condition of every control switch of the form (u^, u), with the 
inequality x' £ d[i]. 



Example 1. Consider the singular slope-parametric rectangular automaton A in 
Figure 2. It has two variables x and y, both of which arc slope-parametric. It is 
initialized since the jump condition for the switch {vq,vi) reassigns y, and the 
jump condition for the switch (vi,V 2 ) reassigns x. The automaton A has the 
value parameter [i] (see Step 1). The initial condition for mode vq has changed 
from 1/7 > 0 to [i] >0 (see Step 2). The slope specification x = 7 in mode V 2 
has been replaced with x = 1, and likewise the slope specifications of the form 
y = 7 (see Step 3a). The slope specification for y in the mode vi is the slope 
parameter 7. Therefore, the invariant y £ 3 in mode vi has been replaced with 
y £ 3[i] (see Step 3b); the guard y £ 5 in the jump condition for the control 
switch (vi,V 2 ) has been changed to y £ 5[i] (see Step 3c); and the assignment 
y := 2 for the switch (vo,vi) has been replaced with y := 2[^] (see Step 3d). 

The assumption of positive slopes makes the exposition simpler, but is not 
necessary. For a slope parameter that is known to be negative, i.e. the invariant 
conditions imply that it is negative, the inequalities are reversed in the transla- 
tion phase, e.g. given a slope specification x = 7 in mode v, the invariant x £ 5 
for mode v would be translated into x £ 5[i], if it were known that 7 < 0. A 
slope parameter that is known to equal 0 may be replaced with the constant 0 . 
In the translated automaton A, the corresponding value parameter is arbitrarily 
set to 0. When the sign of the slope parameter 7 is unknown, the automaton 
can first be replicated into three copies, one for which every invariant condition 
has the additional conjunct 7 > 0 , one for which every invariant condition has 
the additional conjunct 7 < 0 , and another in which the parameter is replaced 
by the constant 0. 

Lemma 1. There exists an algorithm that takes as input an arbitrary initialized 
singular slope-parametric rectangular automaton A and constructs a linear hy- 
brid automaton A and a labeling for its transition system such that (A, A^i) 
IS bisimilar to (A, A^). 

Proof. We show that the construction above yields a bisimilar linear hybrid 
automaton A. Clearly, A is a linear hybrid automaton: all slope parameters have 
been replaced by value parameters. 

Let A,| map each configuration of A to the vector of inverses of the values 
of its value parameters. It remains to show bisimilarity of (A,A^) to (A, A^^). 
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For notational clarity, we rename all variables a; in A as ii, and similarly all 
control modes, control switches, and value parameters. Consider the relation 
G G Qa G Q ^ defined by (v, x, y) G (v, x, y) iff the following three conditions 
hold: 

1. V = V. 

2. For each i G {l..n}, if the flow condition for v specifies Xi = jj for some 

slope parameter then x* = Xiyj, and otherwise X{ — Xi. 

3. For each j G we have yj — l/yj if yj yf 0, and {jj — 0 otherwise. 

For each configuration q of A, dehne a{q) to be the unique conhguration q 
of A such that q ^ q- The idea behind the relationship between q and a{q) is 
simply that whenever a variable x in A has slope equal to a slope parameter 7 
in the control mode v, the variable x has slope 1 in mode v, and the value of 
X in g is equal to the value of x in a{q) multiplied by the value of the 7 in q. 
Furthermore, the slope parameter values in q are inversely related to the value 
parameter values in a{q). 

To verify bisimilarity, we first claim that q q' in A iff a{q) a{q') in A. 

Suppose that q q'. Let a{q) q" . We show that q" — a{q'). Suppose that Xj 
flows at parametric rate jj, i.e. from value a in g to value a + yj6 in q'. Thus 
in a{q'), Xi has value (a + yj5)/yj (since the value of yj remains unchanged), 
which equals afyj + 6, which equals h + 5 since a = byj given that q G a{q). But 
we also have that Xj flows at rate 1 from value b in a{q) to value 6 + if in q". 
Note that the invariant condition in v is satisfied, since the predicate x^ G d is 
equivalent to x^yj G d which is equivalent to Xi G d/yj which is equivalent to 
Xi G dijj, since yj — 1/yj by condition (3) of the definition of G . Hence the value 
of X, matches in q” and a{q'). If the slope of x does not depend on a parameter, 
then its values in q and q' are identical to those in a{q) and q" . Finally, the 
correspondence between the yj and the ijj holds since these values remain fixed 
over control switches. Thus q" equals a{q'), as required. Uniqueness of successors 
under implies the reverse implication that completes the proof of the claim. 

Second, we verify that jump transitions in A and A match. Suppose that 
q A q'. We show that a{q) A a{q'). The case of A simulating A is similar. As 
noted above, if x has parametric slope 7 , x G d holds at g in A precisely when 
X G d[i] holds at a(g) in A. Thus the guard for e is met at g in A iff the guard 

for e is met at o:(g) in A. The value of x may be adjusted by the jump conditions 
so that in every mode in which x has parametric slope 7 , the value of xy in a 
jump successor of a(g) equals the value of x in a jump successor of g, thereby 
satisfying the second condition in the definition of G . For example, whenever 
the variable x is reset within the constant interval [a, 6 ], if the flow of x depends 
on the parameter y^ in the target mode, then the variable x is reset within the 
interval I — [o[;:^] Thus if, in g', x is assigned the value c G [a,b], then 
X may be assigned the matching value c/y^, since it lies in the interval I with 
set to l/yj as dictated by q. The third condition of the definition of G is 
also met since the yj and ijj values are not changed by jump eonditions. Thus 
a{q) A a(g'), as required. 
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Note that initialization requires that the variable x be reset whenever the 
control switch causes a change in the flow of x that involves a slope parameter. 
Otherwise, for instance, in order to capture a conjunct of the form x' — x m. 
moving from a mode in which x has a constant slope (in which case the value 
of X equals the value of x) to one for which it has parametric slope 7 (in which 
case the value of x must equal x/q), we would be required to use a nonlinear 
update of the form x := x[i]. □ 

Theorem 1. Nonemptiness of initialized singular slope-parametric rectangular 
automata is sernddecidahle. 

Proof. Immediate from Lemma 1 and the fact that nonemptiness of linear hybrid 
automata is scmidccidablc. □ 

The translation to linear hybrid automata is of more than theoretical interest. It 
enables us to leverage off existing implementations of model checkers for linear 
hybrid automata, such as HyTech [HHW95,HHW97] and PoLKA [HRP94]. 



3.4 Interval-bounded slopes 

We now show that nonemptiness is semidecidable when the slope specifications 
are intervals, subject to the initialization condition. For this, we use a reduction 
based on mutual similarity, rather than bisimilarity. A singular automaton is a 
singular slope-parametric rectangular automaton for which the set P of param- 
eters is empty. An automaton is strictly initialized if for every control switch 
e = {v, t/), and for every variable x, if the flow specification for x is different in v 
and v' , then x is reassigned to some constant interval by the jump condition of e. 
Henzinger et al. show how to construct a mutually similar singular automaton 
M{A) for any given strictly initialized rectangular automaton A [HKPV95]^. 
The idea is as follows. Suppose that the variable x is subject to a flow condi- 
tion that bounds its slopes in the interval for constants G Q. The 

drifting variable x is replaced with two variables xi and with fixed rates. The 
new variables encode the range of possible values for x as the interval [xi,Xu\. 
The variable xi evolves at rate ki, and the variable at rate fc„. Comparisons 
of the variable x to the constant c in A arc translated into comparisons for xi 
and Xu in M{A). For example, x € c {resp. a: G c) in the guard of a switch is 
translated into xi E c {resp. G c). Assuming that the value of x is unchanged 
by the control switch, the value of {resp. xi) may also need to be updated to c 
by the jump condition to maintain consistency of the bounding range in M{A). 
Whenever x is updated by a control switch, the values of xi and must also 
be updated to reflect the new values of x. 

^ Note that a more general definition of initialization is permitted in this paper, s- 
ince the construction in the proof of Lemma 1 need only produce a linear hybrid 
automaton, whereas in [HKPV95] the decidability results required the translated 
automaton to be a timed automaton. 
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Fig. 3. Rectangular automaton A and a mutually similar singular automaton 
M{A) 



Example 2. Consider the automata A and M{A) in Figure 3. The automaton A 
has two variables x and y, which are replaced by x;, yi, and ?/„ in M{A). 
The ranges of x and y in the initial condition x = 1 A j/ G [1, 3] in A are encoded 
using the upper and lower bounding variables in M[A). The invariant x G 10 for 
mode Vo in A is translated into the invariant X( G 10 in M{A), indicating that 
some value in the range [x;,x„] is less than or equal to 10 provided the lower 
bound X; is less than or equal to 10. Similarly, the guard y G 2 is translated into 
yu G 2, and x = 2 is translated into x; G 2 Ax„ G 2. In A, the flow specification 
for X is different in vq and vi, and therefore the variable x must be reassigned 
into a constant interval by the jump conditions for each switch in A. Consider 
the control switch (vq, vi) in A. There are two corresponding control switches in 
M{A). Both include a reassignment of the xj and x„ variables, reflecting the new 
range of x. The upper switch in the figure also includes the reassignment yi := 2 
when yi is less than 2 , since the range of possible y values immediately after the 
switch must not include any values for y <2. This update is not necessary when 

yi e 2. 

Lemma 2. There is an algorithm that takes as input an arbitrary initialized 
slope-parametric rectangular automaton A and its standard labeling function A .4 
and constructs a linear hybrid automaton B and a labeling Xb for Tb such that 
{A,Xa) and {B,Xb) are mutually similar. 

Proof. Let A be an initialized slope-parametric rectangular automaton. Assume 
that the sign of every slope parameter is specified by every invariant condi- 
tion of A. Then Henzingcr et al.’s construction can be extended to A to yield 
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Fig. 4. M{A), mutually similar to A 



an initialized singular slope-parametric rectangular automaton M (A) such that 
{M{A),Xa) is mutually similar to {A,Xa)- The proof of mutual bisimilarity is 
essentially the same as that of [HKPV95]: the constants in the slope interval 
specifications are replaced by slope parameters of known sign. The interested 
reader is referred to [HKPV95] for details. 

For a slope parameter 7 for which the sign is unknown, the automaton A 
must first be replicated into three copies, one for 7 positive, one for 7 negative, 
and one for equality with zero, just as in the construction for Lemma 1. 

Applying the construction of Lemma 1 to M{A) produces a linear hybrid 
automaton M{A) that together with the labeling that maps to the vector of 
inverse values of the parameters is bisimilar to {M{A), A^i) and mutually similar 
to {A, Xa)- □ 

Example 3. Figure 4 depicts the singular slope-parametric rectangular automa- 
ton M (A) and the linear hybrid automaton M (A) derived from the slope- 
parametric automaton A of Figure 1 by following the constructions in the proof 
of Lemma 2. Automated analysis using HyTech yields the nonemptiness con- 
dition 2[;Aj £ 1 /\ 5[:T] g I for M{A), from which we infer the nonemptiness 
condition 71 G 2 A 72 € 5 for A. 

Theorem 2. Nonemptiness of initialized slope-parametric rectangular automata 
is semidecidable. □ 

Remark 1. The methodology we describe also results in semialgorithmic proce- 
dures for the parametric synthesis problem. If an initialized slope-parametric 
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rectangular automaton is nonempty for a given parameter vector, the procedure 
will eventually output a constraint that includes this vector. Furthermore, if 
the procedure terminates, it gives an exact solution to the parametric synthesis 
problem. 



Remark 2. The analysis methodology above can be extended in various ways. 
The reduction technique applies equally well for automata in which there are ad- 
ditional variables that are linear (they appear only in linear inequalities) and are 
never compared to the slopc-paramctric variables. In particular, the additional 
variables may be value parameters. Furthermore the restriction on flow condi- 
tions can be relaxed to permit endpoints on slope intervals which are not merely 
single slope parameters, but of the form l/(p for a predicate (p in CTnv{r). In 
particular, slope specifications of the form x E [271,672] can be handled. 

In some systems, it is natural to impose additional constraints on the slope 
parameters. For example, a scheduler may choose to allocate its fixed resources 
to one process at a rate fci, and to another at rate /c2, subject to the condition 
that fci + fc2 = 1 [BR97]. Such a condition cannot be directly encoded into an 
initialized slope-parametric rectangular automaton. However, the system may 
still be analyzed via a two-step process. First analyze the system as outlined 
above without regard to the constraint. This determines the combinations of 
values of fci and ^2 for which the system violates its specification. Then, restrict 
these values to those that satisfy fci + /c2 = 1. This second step may be performed 
manually, or via an algebraic tool such as Mathematica. 



4 Undecidability 

It is known that the emptiness problem for initialized rectangular automata is 
decidable [HKPV95]. Here we show that the addition of even a single slope pa- 
rameter renders the emptiness problem undecidable, even for singular automata. 

Theorem 3. The emptiness problem for initialized singular slope-parametric 
rectangular automata with three variables and one slope parameter is undecid- 
able. 

Proof. The halting problem for two-counter machines is undecidable [HU79]. 
Thus it suffices to show that this halting problem reduces to the emptiness 
problem over initialized singular slope-parametric rectangular automata. 

Let M be a two-counter machine with counters C\ and C 2 , and a finite set of 
locations {l\, ... ,ln}. Each move of M shifts control from one location to anoth- 
er, and may either decrement or increment one of the counters. Moves may also 
check whether one of the counters equals 0. The machine M is initially in location 
l\ with the value of both counters being 0. The machine halts if it reaches the fi- 
nal location l„. We construct an initialized singular slope-parametric rectangular 
automaton Am such that Am is nonempty iff M halts. The automaton Am has 
three variables and a single slope parameter. Each counter Ci is encoded by the 
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X2 = 1 X2 := 0 xa = 1 -»■ X2 := 0 xa = 1 *a := 0 





Fig. 5. Decrementing counter Ci while moving from location li to Ij 



variable Xi . There is an additional variable w which is used to implement “wrap- 
ping” of the counter variables, as introduced by Cerans [Cer92], and exploited 
in the undecidability results in [HKPV95]. The slope parameter k, restricted so 
that its value is strictly positive, determines the “step” size of 5 = 1/fc. For each 
program location li of M, there is a control mode li in Am- The location li will 
be reachable in M with the value of C\ being ci and the value of C 2 being C 2 iff 
the configuration {li,x\ — c\5,X2 — C 2 S,w — 0,k) is reachable in Am for some 
k sufficiently large that xi and X 2 are less than 1. Initialization of the program 
location and the counters, and testing for equality with 0 is straightforward. We 
show how to increment and decrement counters along moves. 

To decrement Ci, we use a sequence of transitions whose total duration is 
exactly one time unit. The wrapping variable w times the move. See Figure 5 for 
a decrementing move from location li to location Ij. The value of C 2 , encoded in 
X 2 , is maintained by wrapping X 2 around the value 1. The variable x\, encoding 
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*3 = 1 := 0 = 1 -»• *3 := 0 *2 = 1 := 0 





Fig. 6. Decrementing counter Ci using variables with global rates 



Cl, is also wrapped around 1, but additionally stalls for 5 time units (in mode 
delay-Xi), thereby decrementing its value by 6 by the end of the move. The 5- 
tiine delay is achieved by having xi progress at rate k from a value of 0 to a 
value of 1 in mode delay-Xi, at which point it is reset to 0. This maneuver takes 
S = l/k time units. 

To increment C\, we first use w at rate k to measure 6 time units while 
allowing xi and X 2 to increase at rate 1, and then we decrement X 2 by <5 as 
above. For wrapping to work correctly, we require xi and X2 to remain below 
the wrapping value of 1. Thus we require the slope parameter k to be chosen 
sufficiently large that the increment can be performed whilst keeping both xi and 
X 2 below 1. Since any halting execution involves only finitely many increments, 
this requirement can be met by appropriate choice of fc. □ 
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*3 = fc ^ ®2 := 0 X2 = k X2 ■= 0 X2 = fc -»• *3 := 0 





Fig. 7. Decrementing counter C± in a parametric timed automaton 



Remark 3. The proof above constructs an automaton in which all three variables 
sometimes have rate 1 and sometimes have rate k. In many systems, the rate 
for each variable is global-, it does not vary over control modes. Emptiness can 
be shown to be undecidable for the case of three clocks with global rate 1 and 
one skewed clock with global rate k. In the proof of Theorem 3, durations of 
1/k (such as that occurring in mode delay-Xi in Figure 5) are measured by the 
counters or the wrapping variable advancing at rate k. At all other times, these 
variables have slope 1. The durations of 1/fc may alternatively be obtained using 
an additional skewed clock y that progresses from 0 to 1 at rate k. See Figure 6 
for a decrementing move. 

Remark 4- Parametric timed automata [AHV93] are a subclass of linear hybrid 
automata, in which all variables are partitioned into two sets: clocks Xc, which 
have a global rate of 1, and value parameters Xp. In addition, every invariant, 
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Fig. 8. Timing diagram for Manchester encoding of 10011 



initial, final, guard, and jump condition is a parametric rectangular predicate 
in VTZ{Xc,Xp). Thus clock values can be compared to value parameters. It 
can be shown that the emptiness problem is undecidable for parametric timed 
automata having three clocks and one parameter, c.f. undecidability is proven 
in [AHV93] for the case of three elocks and six parameters. The eonstruction 
in the proof of Theorem 3 can be modified to encode counter values directly as 
clock values, and to wrap clocks around a potentially large- valued parameter k, 
instead of the constant 1. The delay of step size 1 is obtained by allowing a clock 
to advance from 0 to 1. See Figure 7 for a move that decrements C\. 

5 Example: an Audio Control Protocol 

We perform analysis of the slope parameters in a timing-based bit-level commu- 
nication protocol. The protocol is used by Philips Electronics N.V. for sending 
messages between stereo components [BPV94]. It is part of a local area net- 
work used by control programs that provide integrated features such as system 
activation or CD-to-cassette dubbing in response to a single button press. The 
single-sender single-receiver version of the protocol was first formally specified 
and verified by Bosscher et al. [BPV94]. They also analyzed the tolerable clock 
drift. The drifting rate of both the sender’s and the receiver’s clocks were speci- 
fied using the bounding interval [1 (8 e, 1 -|- e]. The protocol works correctly iff the 
value of the parameter e is less than 1/17. This fact was established manually 
in [BPV94], and later proof-checked by Griffioen in the Larch Prover [Gri95]. 
The error bound was obtained automatically in [HW95]. That analysis relied 
critically on the fact that both clocks were subject to a common error tolerance, 
enabling a global transformation of the time scale. Here, we use a more general 
slope specification, using four independent parameters, one for the lower and 
upper bounding rates for each of the sender and the receiver. 

5.1 Protocol description 

The protocol uses Manchester encoding to transmit sequences of bits. For a full 
description of the protocol, see [BPV94]. The time line is divided into equal- 
width slots of duration 4Q, and bit values are transmitted in the middle of time 
slots. A 1 bit is sent as a rise in voltage, and a 0 bit as a fall in voltage. To send 
the same bit in two consecutive time slots, there is a change in voltage at the 




408 



II. Wong-Toi 



k:=l-k 



k := 1 — k 
inputg 




inputs 



Fig. 9. Message-generation automaton 



intermediate time slot boundary, as depicted at time 8Q in the encoding of 10011 
that appears in Figure 8. Message must begin with a 1 and have length at least 
three. There is also a silence of at least lOQ between the end of the transmission 
of one message and the start of the next. The protocol is complicated by the fact 
that downgoing edges cannot be reliably detected, and so the receiver uses only 
upgoing edges to decode the message. Because the downgoing edge of a final 0 
bit is not detected, there arises an ambiguity between messages ending in 1 and 
in 10. To solve this problem, there is an additional requirement that messages 
be either odd in length, or end in 00. Furthermore, although the sender and 
receiver use an a priori fixed width for the time slots, their clocks are subject 
to independent drift, leading to possible misinterpretation of the bit stream. For 
our correctness criterion, wo require that any arbitrary-length legal sequence of 
bits input to the sending component must be correctly received and output by 
the receiving component. 



5.2 Parametric analysis of clock drift 

The protocol is modeled using two (initialized) slope-parametric rectangular 
automata (one for the sender and one for the receiver) and two linear hybrid 
automata (one for generating input sequences and a monitor for checking the 
output). For completeness, the automata appear in Figures 9, 10, 11 , and 12. 
Slope-parametric automata can be composed to form a slope-parametric au- 
tomaton modeling their parallel composition. To this end, the automata are 
embellished with event labels on control switches, and control switches in one 
automaton A having an event label that is common to some other automaton 
B can only occur when synchronized with a control switch in B with the same 
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event label. Modes in the product are the cartesian product of modes in the 
component automata, and invariants, final, initial, flow, and jump conditions 
arc the conjunctions of those in the components. The formal definition of com- 
position is not given here, but it is materially the same as that for linear hybrid 
automata [ACH+95]. 

The input bit stream is nondeterministically generated on-the-fly by the 
message-generation automaton, which uses the variable k to record the pari- 
ty of the length of the message. The bits that are generated are passed to the 
sender automaton via the synchronized event labels inputQ (for a 0 bit) and 
input I (for a 1 bit). The sender transmits timely Down and Up signals using 
the Manchester encoding described above. Its drifting clock x has a global rate 
interval [U, Ux], for positive slope parameters lx and Ux- The receiver automaton 
uses the variable y to model its drifting clock with slope speciheation y G [ly, Uy] 
for positive slope parameters ly and Uy. The variable m is used to record the 
parity of the message received. The receiver recognizes only the Up signals. Up- 
on receiving an Up signal, it determines the slot boundary (or middle of a time 
slot) at which it occurs and updates its copy of the message. A direct modeling 
of a receiver, which adds bits to its own copy of the message and outputs the 
entire sequence once transmission ends, would cause the symbolic analysis of the 
reachable state space to be nonterminating, due to input sequences of arbitrary 
length. The receiver instead outputs bits as it receives them, via the event labels 
output^ and outputi. This modeling choice does not affect the correctness of the 
protocol. 

The monitor automaton verifies that the receiver outputs the same message 
that is input to the sender by comparing the sequence of bits output by the 
receiver to the sequence produced by the message-generation automaton. The 
automaton uses the variables leng and c. The variable leng models the number 
of bits currently transmitted by the sender but not yet acknowledged by the 
receiver. The variable c models the numeric value of the binary encoding of those 
bits. Whenever the receiver automaton takes a control switch labeled output q 
or outputi, the monitor checks the intended output bit-value with the value 
of the earliest bit sent but not yet acknowledged. If the output bit does not 
correctly match the input bit, the monitor enters its error control mode. The 
monitor also verifies that the entire bit stream is received by its control switch 
that checks that there arc no remaining bits when the receiver takes the control 
switch labeled done. The final states of the monitor automaton are all states in 
the error control mode. 

The mutually bisimilar linear hybrid automaton of the product of the four 
component automata has 438 control modes, four parameters, eight other vari- 
ables, and 1468 transitions. The initial condition includes the restriction 0 < 
lx ^ Ux U 0 < ly E Uy. Parametric analysis in HyTech takes 36 iterations in 
265 seconds on a Sparcstation 5 with 32 MB RAM. The resulting conditions for 
correctness are output as 

8ilx < 9iuy & 7ily < 8iux 

& iuy <= ily & iux <= ilx 
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where ilx denotes the parameter [|^], etc. Hence, the system is correct iff ?>Uy < 
A 7ux < Sly A D Ux Aly U Uy. In particular, with lx — ly and Ux — Uy, this 
gives the previously known result that e be less than 1/17, and additionally, for 
an exact sender [resp. receiver) clock, the receiver [resp. sender) clock may drift 
in the interval [|, |] {resp. [|, f]). 

6 Conclusions 

We show how parametric slope analysis can be performed for initialized slope- 
parametric rectangular automata. The reduction method allows us to use ex- 
isting model checkers for linear hybrid automata to analyze slope parameters. 
The emptiness problem is proven undecidable even for initialized singular slope- 
parametric rectangular automata. 

Despite the formidable theoretical limitations to parametric slope analysis, 
our investigation of the audio control protocol illustrates that the analysis pro- 
cedure may be useful in practice. 

Acknowledgement. Thanks to Vlad Rusu for helpful comments on this paper 
and its relationship to his work. 
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Fig. 11. Receiver automaton 



c := 2c, leng := leng + 1 c := 2c -)- 1, leng := leng + 1 

inputg input^ 




Fig. 12. Monitor automaton 
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Abstract. This paper presents a control framework for interval tempo- 
ral systems, in particular systems modeled by expressions of Duration 
Calculus. Duration Calculus is a formal specification language for real- 
time systems. Its distinguishing feature is to formalize and reason about 
interval properties of discrete states of systems. In this paper, interval 
temporal systems are taken as dynamic systems equipped with a state 
space, and a state transition structure. A control structure is adjoined to 
such systems, and this makes it possible to restrict the systems behaviors 
so that the closed systems behaviors can satisfy the required specifica- 
tions. Some typical control issues are solved within this framework. 



1 Introduction 

The research of hybrid systems is very active in recent years, stimulated by the 
wide use of hybrid systems in industries, and the development of control theory 
and computer science. Structurally, a hybrid system contains two levels and an 
interface between them, the high level is a decision maker whose behaviors are 
driven by events, and the low level is a process to be controlled which evolves in 
real time. 

The research of hybrid systems could be started from different departure 
points. One can start from the theories of conventional control theory which 
deals with time evolving variables only, or from discrete event control theory 
which deals sequential and parallel behaviors of events (or symbols), or even 
initialize a theory which can deal with both kinds of variables. 

It is hard to say which is the best method (at least so far!). We are going to 
depart from theories of real time systems in computer science, and extend them 

* This work was supported by the 863 Hi-Tech Programme of China. 
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gradually, and eventually incorporate continuous variables as well. It should be 
noted that there is no particular reason for us to depart from this point and not 
the other. 

Duration Calculus, a real time specification language, will be adopted in our 
methods. Duration Calculus is an extension of interval temporal logic, invented 
by Zhou, Hoare and Ravn [5] . The calculus formalizes integrals of Boolean func- 
tions over time intervals, and it can specify and reason about timing and logical 
constraints on discrete states of a system. 

As a first step toward hybrid control theory, we shall establish a control 
structure for systems modeled by Duration Calculus expressions. To this end, in 
this paper we shall introduce a control mechanism to a class of real-time systems 
modeled by Duration Calculus. The behaviors of the system are specified by 
Duration Calculus formulae. Various control concepts such as controllability, 
controller synthesis, and optimal control, can be defined or solved for a given 
behavior. 

We begin with a modeling technique in Duration Calculus, based this mod- 
eling technique, a control mechanism is introduced. 



2 Duration Calculus 

Duration Calculus was introduced in [5] as a notation for specifying and reason- 
ing about timing and logical properties of discrete states in dynamical systems. 
Its distinguish feature is formalizing and reasoning about duration of states with- 
in any time interval, without explicit mention of absolute time. There are several 
extensions of Duration Calculus, for example Extended Duration Calculus [6], 
which was motivated to additionally capture properties of piecewise continuous 
states of hybrid systems. Duration Calculus has been used to specify and verify 
requirements and designs for several examples such as gas burner [5]. 

In this section, we will briefly introduce Duration Calculus. For more detail, 
please refer to [5]. For those who arc familiar with Duration Calculus, please 
skip over this section. 

Syntax 

Duration Calculus is an extension of Interval Temporal Logic(ITL), it retains 
the alphabet of symbols of ITL by deleting the temporal variables and by adding: 

— state variables X,Y, Z, . . and, 

— the special symbols 0 and 1. 

States: 

The set of states is generated by the rules: 

— 0,1 and every state variables are states, 
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— if P, Q are states, so are {^P) and (P V Q). 

Any state in Duration Calculus is assumed to be of finite variability, i.e., within 
any observation interval, any state can change at most finite times. 

Durations: 

Let P be a state, the duration of P over an observation interval [b, e] is 
defined by 

JP^JlP{t)dt. 

It follows that f 1 — I, where l(— e ® 6) represents the length of interval. 

A state P can be lifted to a simple predicate [P] in ITL. \P] is defined as 

[P1 = (/P = /1)A(/1>0), 

which means that P holds almost everywhere in a proper observation interval. 

Terms: 



— I, J P and global variables x are terms, 

— if ri, C 2 , . . . , are terms, and /" is an n-ary function letter, then /”(ri, r 2 , 
. . ., Tn) is a term. 



Formulae: 

Atomic duration formulae are built from duration terms. If A" is an n-ary 
predicate letter and ri, r 2 , . . . , r„ are terms, then A"(ri, r 2 , . . . , r„) is an atomic 
duration formula. The set of duration formulae is generated by the rules: 

— true, false and atomic duration formulae are duration formulae, 

— if Pi, D 2 are duration formulae, so are ~^Di, D\ V D 2 , Di \ D 2 , and (Vx)Pi, 
where x is any global variable. 



are used: 



The following abbreviations 

\]={l = 0) 

true — {1^0) 
false = {-^true) 

OP = true-, P; true 
□P = 



the empty interval 
holds for any interval 
holds for no interval 
P holds for some subinterval 
P holds within any subinterval 



Proof System 

These axioms are from ITL, and arc retained in Duration Calculus. 

1. Monotonicity. If D\ ^ D[ and D 2 ^ D '2 then Pi; P 2 => D[-, D' 2 . 

2. Associativity: (Pi; P2); P3 Pi; (P2; P3). 

3. Zero: P; false ^ false and false-, D false. 
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4. Unit: D; D; \ ] [];£)• 

5. Distributivity: 

{D\ V D2)] D3 => {Di; Ds) V D2; -D3, 

Di', {D2 V D3) ^ (£>i; D2) V D\] D3. 

6 . Continuum: I — r + s ^ {I — r): {I — s), {r, s ^ 0 ). 

In addtion to these axioms, Duration Calculus has the following axioms about 
duration of states. 

1 . /0 = 0 . 

2 . For an arbitrary state P, 

JP ^ 0. 

3 . For arbitrary states P and Q, 

fP + fQ^f(PVQ) + f(PAQ). 

4. Let P be a state and r, s non-negative reals, 

(/P = r + s)o(/P = s);(/P = r). 

In addition to these axioms, there are two induction rules in Duration Cal- 
culus: 

Let X denote a formula letter occurring in the formula R{X) and let P be a 
state. 

Feed-ward Rule: If i?(M) holds, and R{X V X; [P]) A R{X V X; [^P]) is 
provable from R{X), then R{true) holds. 

Backward Rule: If i?(M) holds, and P(X V [P];X) A P(X V h^l;X) is 
provable from P(X), then R{true) holds. 

The axioms and the induction rules can be shown to constitute a sound and 
(relative) complete formal system of durations [ 1 ]. 

Some Useful Theorems : 

Using the axioms and the induction rules, we can prove readily properties 
like: 

Lemma 1. For an arbitrary state P 

SP + hP = ^^- 

2. fp^ [1. 

5. [ 11 V n . 

4- For any state P , 

[P] ^ {hP = 0 ), 

5 . For a state P and non-negative reals r, s, t and u, 

(7 — I J P ^ s); {t ^ jP-<u)^{r + t-<JP^s + u). 
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3 Modeling in Duration Calculus 

3.1 A Modeling Framework 

System models are the basis for the development of control theory. In this section, 
we shall give a modeling framework for interval temporal systems which are 
described by Duration Calculus. 

We assume that a system is of finite states. A system to be controlled is a 
four-tuple of {S, So, G, 5}, where S is the set of states, S — {Si, 82 ,^^ Sn}, 
So is the initial state, G is the set of so called guards which cause the state 
transitions, and 5 is the transition function, it is a (possibly partial) function 
from S -> G to S. They satisfy the following constraints: 

^ State Completeness The states in S can completely capture all behaviors 
of the considered system. Formally, for any given observed interval, it is either a 
point interval or a non-point interval where at least one of the states in S holds 
within that interval. 




^ Mutual Exclusiveness We require that the number of states in S is mini- 
mum. In another word, the states in S arc mutually exclusive. More formally, for 
any given observed interval, it is either a point interval or a non-point interval 
where there is not such a case that any two or more states can describe the 
system simultaneously. 

V Uo ( V A r^,i) ) • 



Transitional Constraints The transitional constraints define a subset of 
state transitions which are or are not possible, i.e., the transitions that are or 
are not defined by transition function S. 

Initial State So is the initial state, then for any non-point interval, the 
following formula holds. 

[S'o] ; true. 



3.2 Example: Reservoir 

A reservoir is modeled by a transitional system with three states: low, m,edium, 
and high, the initial state is low. The set of guards includes and which 
corresponds the rising and dropping of water level respectively, d is a partial 
function: 5{low,g^) = medium, 6 {medium, g^) = high, 6 {high, g~) = medium, 
5{medium,g^) = low. 
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^ State Com,pleteness All possible states of the reservoir can be represented 
by the three states: low, medium or high. 




n 

\low V medium V high] 



Mutual Exclusiveness The three states: low, medium and high are not 
only enough to describe the behaviors of the reservoir, but also a minimum 
combination to determine any status of the reservoir. 



V 



-O 



V 




\low] A \medium?\ 
[med'mm] A \hig}i\ 
\high] A \low] 



\ 

/ 



^ Transitional Constraints There are several transitional constraints with 
the reservoir, first there are no such transitions as from low directly to high or 
vise versa. 

Consi = V ^ f\/ f \high]\\ . 

[\high]-,\low])j J 

Secondly, it is physically not possible for a transition from lower level water to 
higher level water if the water level is decreasing, or a transition from higher 
level water to lower level water if the water level is rising. 





/n 






\ 


V 




/ 


/ \low~\ A \g~]; \medium] \ 


\ 




V 


\m,ed,ium] A \high~\ 

\medium~\ A [5^]; \low~\ 






1 


1 


y [high] A \medium] j 


// 



^ Initial State The initial state of the reservoir is low. 

Conss = I" 1 V (\low];true). 

Let Consres — Consi A Cons 2 A Conss. 



3.3 Model 

We shall first define the space of all possible behaviors of a system, then define 
open loop behavior by taking transitional and initial constraints into considera- 
tion, finally define closed loop behavior by further taking the constraints imposed 
by the designed controller into consideration. 

A system has order n if one needs at least n states to uniquely determine the 
status of the system. All possible behaviors of a system are captured by universal 
behavior which is defined below. 
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Definition 1. Suppose a system has n states S',(z S n). The universal behavior 
(specified hy UB) of this system is defined as 







Definition 2. The open loop behavior (specified by OB) is defined as the those 
behaviors of universal behaviors that satisfy transitional constraints and initial 
constraints. Formally, 

OB = UB A Cons, 
where Cons is transitional and initial constrains. 

Definition 3. The closed loop behavior (specified by CB) is defined as those 
open loop behaviors that satisfy controller constraints. Formally 

CB — OB A Cont, 

where Cont is the constraint defined by the designed controller. 



3.4 Example: Reservoir (Cont.) 



The reservoir has three states: low, m,edium, and high, and they are mutually 
exclusive. Therefore the order of the reservoir is 3. 

The universal behavior of the reservoir is 



UBres = \/ 

We can prove that the open loop behavior of the reservoir is 



/n 


f \low] 


\ 


; > 0;V 


\ medium) 




V 


V \high) y 


u 



OBres 



/n 

\low) 

[/ow]; V 

V 



\ 

/ 1 ^ 0; \medium] A \low) \ 

I -1 0; \low~\ A \m,edium] 

I > 0; \high] A [g”]; \medium\ 

\l -I 0; \medium] A \g^]; \high] j j 



Theorem 1. The open loop behavior of the reservoir is specified by OBres, ie-, 

U Bres A ConSres AA OBres- 



In order to prove this theorem, we first prove the following lemma. 
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Lemma 2. 
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V )) 



Proof of Lemma 2: 



This is proved case by case. First let’s prove ConSres A {OBres', 
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OBres ' 

Similarly we can prove 



ConSres A (OBres'i [medium]) OBres, 
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and 



ConSres A [OBres] \high]) ^ OBres- 



Proof of Theorem 1: 

It is obvious that OBres u Bres^ConSres, now let’s prove U Bres^ConSres- 

Let R{X) = UBres A ConSri-s A X OBres- 

It is obvious that i?([ ]) holds. Now let’s assume that R{X) holds, we intend 
to proof that R{X: [lore]), R{X; \mediurn]) and R{X; \high]) all hold. 



UBres A ConSres A (X; \low']) (distr. of A over ;) 

{U Bres A ConSres A X)\ [U Bres A ConSres A \loW~\ ) {R{X) holds) 

OBres ] {UBres A CoTlSres A \low'] ) 

OBres] \lovj'] (Lemma 2) 

OBres 



Similarly we can prove that R{X] {medium]) and R{X\ \high~]) hold. 

The aim of control design is to avoid the reservoir’s entering high state. So 
a specification of control is simply to disable a transition from medium to high. 
Namely for any given interval, there is not such a subinterval where there is a 
transition from medium to high. In Duration Calculus, this can be specified as 

Contres — ~^C'{\medium] A (v > 0); \high)). 

The closed loop behavior of the reservoir is 

\ 



(\ 



CBres — \/ 



V 



\low~\ 

\low]-,\f 



0; {medium,) A {v < 0); {low) 
0; {low) A {v > 0); {medium) 



Theorem 2. 

OBres A Contres AA CBres 

Before we give a formal proof to this theorem, we prove the following lemma. 

Lemma 3. 

{low) ; / > 0; {high) => {low) ; 1 -i 0; {medium) A {v > 0); {high) . 

Proof of Lemma 3: 

We use induction rule to give a proof for this lemma. Let 
R{X) — {low) ;l > 0 A X; {high) {low) ; 1 ^ 0; {medium) A (v > 0); {high) . 

i) When X = [ ], the right hand side of ]) is false, thus P([ ]) holds. 
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ii) Suppose holds, i.e. 

\low] ;l > 0 A X; \high] ^ \low ] ; / ^ 0; \medium \ A {v > 0); [high] . 
Let’s prove that R{X; \low]), R{X, \medium\) and R{X; \high]) hold. 



ConSr 



ConSr 



and 



R{X) 



\lovj \ ; 


1 > 


false, 




\low ~\ ; 


1 > 


\low ] ; 


1 > 


\low ] ; 


1 > 


\low ] ; 


1 > 


\low ] ; 


1 > 


\low ~\ ; 


1 



l^ljy,\h^gh^ 

\low ] ; / > 0 A (X; \mediurn] A (u > 0); \high] , 



Proof of Theorem 2: 

Since there is not such a subinterval where there is transition between m,edium, 
and high, as defined by the controller, it is clearly that the theorem holds. 



4 Control of Interval Temporal Systems 

A control system is a system including a plant, an executer and a controller, as 
depicted in Fig. 1. The plant is the part which is supposed to be controlled, the 
executer is the part through which controller can exert control actions to the 
plant, and the controller is a set of control laws which define how the executer 
may behave. 




Fig. 1. Block diagram of a control system 



4.1 Guard and Coguard 

Suppose P, Q are two states of a plant. For simplicity, denote a transition from 
state P to state Q of a system as P ^ Q. 
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Definition 4. g ih a guard of P ^ Q if P ^ Q can occur when g holds, and 
can not occur otherwise. 

Denote the set of all guards of P ^ Q as Q{P Q). By definition, it is 
obviously that if g is a guard of the transition from P to Q, then a transition 
from P to Q can not occur if ->g holds at the same time when the plant is at 
state P, and if h implies g, then h is also a guard of P ^ Q, and Q{P Q) is 
closed under A and V. 

Theorem 3. Suppose f,g & G{P ^ Q), 

1. [P] A (^g); \Q] = false 

2. for any h ^ g, h e Q{P Q). 

3. Q{P Q) is closed under A and V, i.e., both fAg and fWg are inQ{P Q). 
We introduce the set of all guards of plant V\ 

SiV) = { g I g is a guard of Pi Pj,i / j, i,j G n}. 

We assume that any two elements in G{P) are different from each other, i.e., 
Vgi,g 2 G G{V),gi ^ g 2 - 

Example: Reservoir (Cont.) 

The state transitions of the reservoir are decided by three guards: transitions 
from low to normal, normal to high are guarded by u > 0, transitions from 
high to normal, normal to high are guarded by u < 0, and the stability of 
water level is indicated by v = 0. 

G{Res) = {g+,g“,go}, 

where g+ = (u > 0),g^ = (u < 0), go = (u = 0). 

Definition 5. A guard that is emitted by state transitions of the executer is 
defined as a coguard. 

Suppose E\,E 2 are two states of the executer £, we can then similarly define 
the set of coguards of E\ E 2 and the set of all coguards of £ as CG{E\ 
E 2 ),CG{£), respectively. 

An executer is a map from all its possible states to coguards. 

Example: Reservoir (Cont.) 

The executer of the reservoir includes two valves, each valve has two states: 
on and off. 

\EVoffADVoff]^\go], 

\FVonADVofn ^ Ig+l, 

\EVonADVon] ^ \ go], 

\FVoff A DVon] ^ \g-], 
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where FVoff (FVon) indicates feeding vo,lve is off (on), DVoff (DVon) in- 
dicates discharging valve is off (on). Therefore 

Cg{Valves) = \go,9^,g~}- 



4.2 Controllability 

Definition 6. A behavior B is controllable with respect to OB if and only if 

1. B ^ OB, and 

2 . g{oB!B) - cg{£). 



Theorem f. A necessary condition that B is controllable with respect to OB is 

1. g{B) -I g{OB), and 

2. g{OB/B) - cg{£), 



Theorem 5. A behavior B is controllable if and only if there exists controller 
C specified by Cont such that 

OB A Cont AA B i.e., CB AA B. 

Proof. Necessity: Let g G g{OB / B) , ie., the occurrence of g will invalidate B. 
Therefore we should have g disabled. 

According to controllability, we have g{OB/B) ^ Cg{£), therefore, there 
exist state P, Q of the plant, and states C, D of the executer, so that 

gegiP^Q), g&g{C^D). 

Obvious we can make the controller to disable g if the system is at the state P. 

\P^hg)]- 

Thus we can construct a controller which can disable any g G g{OB/B) at an 
appropriate state of the plant. 

Sufficiency: Suppose there exists a controller specified by Cont such that 
OBACont-e^B i.e., CBaaB. 

1) It is easy to sec that B OB. 

2) Suppose there exists g G g{OB / B), but g ^ Cg(£), which implies that 
there is behavior which invalidates B and is out of the control of controller, this 
contradicts the existence of controller. 
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4.3 Optimal Control 

We introduce the set of all open loop behaviors that are controllable. 

C{V) — {B OB I B is controllable }. 

It will be shown that the siipremum alway exits in C{P). 



Theorem 6. C{P) is nonempty and is closed under unions. The supremum 
(aupC{P)) of C{P) is inC{P), and 

OB — supC(P). 

Proof. Since OB is always controllable, C{P) is nonem,pty. Let B1.B2 C C{P), 
then 

Bi ^ OB, 

g{OBlB,)^Cg{£), (z = l,2). 



We then have 
and 



B\ V B2 OB 



g{OBI{Br V B2)) - g(OBIBt) V g[pBIB2) - cg{z), 

which implies that B\ V B2 is controllable. 

Finally we have, 

sup g{B) = \J {B I B OB, B is controllable with respect to OB } , 

since OB is controllable and any controllable behavior is a subset of OB, there- 
fore 

sup g{B) = OB. 



Theorem 1. Ifg[OB) -1 Cg{8), then any behavior B that B OB is control- 
lable. 

Proof. Since g{OB/B) -< g{OB) -1 Cg{£). 

Example: Reservoir (Cont.) 

The reservior is controllable. 

Example: A Modified Reservoir 

Suppose the reservior has only a discharge valve and has no feeding valve, 
and the requirement is to get the water level to continuously rise. Then it is 
uncontrollable since 

g{Resh^B) = {.g+}, Cg{£) = {go,. 9"}, 



and 



g{Res A -iR) / Cg{£). 
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5 Conclusions 



In this paper, we have presented a control theory for systems modeled by Du- 
ration Calculus. By adjoining control structure to these systems, it might be 
possible to restrict systems behaviors to some certain degree so that the closed 
behaviors of systems can satisfy the required specifications. A modeling frame- 
work of interval temporal systems formalized by Duration Calculus is presented, 
based on which a control structure is introduced for this class of systems. Various 
control problems are studied in this paper, such as controllability, synthesis of 
controller, and optimal control. 

Our work was inspired by the idea of Wonham and Ramadge in their pi- 
oneer work in [2,3]. They considered discrete event systems modeled by finite 
automata, and took an automaton as a generator of formal languages which 
describe the behaviors of a dynamical system. In the framework proposed by 
Ramadge and Wonham(often referred to as RW), the set of events which cause 
the transitions of states are classified into controllable and uncontrollable sub- 
sets. A controller is designed so that it may disable controllable events to meet 
the system requirements. A main difference between our control structure and 
RW is that we do not give such a classification, instead we define the guards 
of the plant and coguards of the executer, controllability is defined in terms of 
the relationship between the set of guards and the set of coguards. Another ma- 
jor diflercnce is that our framework is for interval temporal systems, namely we 
consider temporal properties right at the stage of the initialization of our theory. 

Like the same direction as it was with the classical control theory, many con- 
trol problems are still remained to be researched. One of the important features 
of control systems is of partial observation, in our case, when some of guards of 
the plant are not observable. Currently, we have only considered that the plant 
is completely observed. 

Duration Calculus is a highly potential tool for interval temporal systems, it 
has powerful expressiveness as well as powerful induction rules. Several exten- 
sions of Duration Calculus have been developed, capturing the various features 
of real time systems. Extended Duration Calculus [6] was extended to accom- 
modate piecc-wisc continuous properties of hybrid systems mixed with both 
discrete variables and continuous variables. Mean Value Calculus [7] was intend- 
ed to include point values of Boolean functions such as 5-functions which are 
often used to represent instant actions. Formal speciheation and verification of 
hybrid systems based on Extended Duration Calculus has been carried on by 
some researchers [4] , which indicated some brightness of extending current work 
to hybrid systems settings. We leave the extensions of control for other members 
of Duration Calculi [8] as our future work. 
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Abstract. We describe the Phase-Space Nonlinear Control Toolbox, a 
suite of computational tools for synthesizing and evaluating control laws 
for a broad class of nonlinear dynamical systems. The Toolbox com- 
prises computational algorithms for identifying optimal control reference 
trajectories in the phase space of dynamical systems and experimen- 
tal methods for evaluating performance of the control laws. These algo- 
rithms combine knowledge of the geometric theory of modern nonlinear 
dynamical systems with efficient computational methods for geometric 
reasoning and graph search; they define the properties of controllability 
and robustness in terms of phase-space geometric structures and exploit 
the phase-space neighborhood adjacencies to obtain computational ef- 
hciency. Compared to the traditional analytic control design methods, 
the phase-space based control synthesis and evaluation rely on high- 
performance computational techniques and are applicable to physical 
systems operating in large nonlinear regimes. Using a proof-of-concept 
physical experiment for stabilizing a nonlinear magnetic levitation sys- 
tem, we have successfully demonstrated the feasibility of the phase-space 
control technology. 



1 Introduction 

Many physical systems such as man-made electro-mechanical systems operate 
in large nonlinear regimes. These systems often exhibit extremely complex be- 
haviors that defy conventional analytical analysis and numerical simulations. 

This paper describes a suite of computational methods for nonlinear control 
synthesis and analysis that computationally explore the phase space of dynamical 
systems guided by domain knowledge of dynamical systems and control theo- 
ry. We have constructed a proof-of-concept physical experiment for a nonlinear 
magnetic levitation (maglev) control system and demonstrated the feasibility of 
the phase-space control technology. The maglev project serves as a testbed for 
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developing practical phase-space based control algorithms, performance criteria, 
and evaluation methods. 

The phase-space based control has several important advantages compared 
to conventional control design methods. It relies on a computational character- 
ization of phase-space geometry and exploits the geometric knowledge to guide 
control planning and execution. It captures global dynamical behaviors and re- 
quires no linear approximation. It explores a much larger space of possible control 
strategies than linear control methods do. It trades computational resources with 
control performance. The important stability and robustness properties of a con- 
trol system are geometrically interpreted in phase space and can be operationally 
verified using the geometric models. We expect the phase-space control meth- 
ods to complement conventional techniques and find niches where conventional 
methods arc not applicable. 



2 Phase-Space Control Synthesis 

Poincare’s geometric method of modern dynamical systems provides the theo- 
retical basis for the phase-space analysis and synthesis of nonlinear dynamics [7] . 
A phase space for a dynamical system is spanned by the independent state vari- 
ables of the system. For instance, a swinging planar pendulum’s phase space is 
a two-dimensional plane of position versus velocity. In phase space, important 
qualitative behaviors of dynamical systems are characterized by the geometric 
features of the space such as points, curves, surfaces, and volumes — equilibrium 
points, limit cycles, stability regions, trajectory flows, and their spatial arrange- 
ment — that can be extracted, identified, and exploited through computational 
means. 

Control theory and engineering provide a body of tools for designing linear 
control systems. Mathematical results on stability and controllability of linear 
systems have successfully guided practical implementations of linear controller- 
s. In contrast, nonlinear control lacks general methods that provide a unified 
treatment of and approach to a wide class of nonlinear systems. Recent theoret- 
ical work on the controllability of nonlinear systems that employs a differential 
geometric approach is still far from being practical [11]. Conventional analyti- 
cal methods have two important limitations: they require accurate models and 
labor-intensive simulation and calibration. 

As an example of a nonlinear control system, we examine the well-studied 
control problem for legged robots. The hopping motion of a legged robot can 
be described as a limit cycle in a phase space of position and velocity, with 
distinct points (events) delineating phases such as lift-off, touch-down, or flight 
phase of the motion [15]. The behaviors of the robot are characterized by the 
number and shapes of the limit cycle. After having successfully designed several 
generations of legged robots, Marc Raibert anticipated the need for and the 
possibility of automatic control synthesis methods for designing more complex 
legged machines such as gymnastic robots: 
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The strategy we chose is based on several decisions... Each of these 
decisions was made by hnmans based on knowledge of the mechanics of 
the problem and intuition. It is not hard to imagine that future control 
systems may be able to formulate strategies such as these automatical- 
ly [9]. 

The phase-space approach provides a systematic way to explore the control 
spaces of physical systems. Radically different control behaviors can be automat- 
ically synthesized and evaluated by systematically varying operating conditions 
and initial states of the robots. For instance, by varying the magnitude and 
length of the thrust for the robot, distinct trajectories may be generated, exam- 
ined, and selected according to specified optimality criteria. 



2.1 Overview of phase-space control 

Combining the phase-space geometric description of dynamics with mathemati- 
cal characterization of stability and controllability from control theory, we have 
developed computational algorithms for aggregating, classifying, and searching 
for optimal control reference trajectories [2,21,22]. The main ideas of the phase- 
space control synthesis are illustrated here using the spatial aggregation frame- 
work [20]. For simplicity, we consider the stabilization control problem where 
the control objective is to find reference trajectories to steer the system towards 
a prespecified goal state in phase space. Given the phase-space data descriptions 
as input, the spatial aggregation operators aggregate, classify, and search, 
transform the low-level field data into global control policies by exploiting the 
spatial-temporal neighborhood structures of the vector fields. 

— Aggregation: A phase-space vector field, parameterized for a particular 
control action, are aggregated into a neighborhood graph explicitly encod- 
ing adjacencies for a grid of phase-space cellular regions. At the higher level, 
neighborhood graphs parameterized for different control actions are aggre- 
gated to a composite neighborhood graph. Two cells are adjacent if there is 
a trajectory that connects the cells under a certain control action. 

The nodes of neighborhood graph can further record probabilities of tran- 
sition due to discretization or uncertainties in measurements. It is possible 
to have multiple edges directly connecting two cells due to different control 
actions. The edges of the graph can be weighed according to the quality of 
control. For instance, the weight can measure the amount of control resonrees 
such as time or control energy consumed while making the transition. 

— Classification: A neighborhood graph is classified into equivalence classes of 
cells. At the first level, the cells are grouped into behavioral classes according 
to robustness or other control concerns. Information such as the likelihood 
of being perturbed off the path is valuable for the search procedure at the 
higher level. 

At the next level, the cells in the composite neighborhood graph are classified 
into two classes: one corresponds to the collection of cells in the controllable 
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region, i.e., each cell has a path to the goal state; the other class comprises 
cells that are disconnected from the goal. A more sophisticated classification 
scheme labels groups of cells according to the probability of making it to 
the goal starting at different initial states in a cell. We have implemented a 
version that classifies regions into controllable, marginally controllable, and 
uncontrollable types. 

— Search: The classified phase-space neighborhood structure is then searched 
for optimal control trajectories. A control reference trajectory (or a control 
law) consists of a sequence of cells and the associated control actions that 
induce transitions from cell to cell. 

We have implemented an iterative algorithm that computes optimal paths 
based on a given optimality function (e.g. based on response time or robust- 
ness). Additionally, we have implemented a dynamic programming algorithm 
that efficiently identifies shortest paths to the goal in the graph according 
to various definitions of distance functions. 

The control paths can be computed using the system model or state mea- 
surements taken directly from the system. The paths are then stored in a 
table for efficient run-time retrieval. Sub-optimal control paths can also be 
retained to permit graceful control degradation in the presence of uncertain- 
ties. The controller can trade off the amount of computation with run-time 
control quality. 

The main steps of aggregation, classification, and search are summarized in 
Figure 1. 



2.2 Geometric interpretation of control performance 

We formalize the important control properties for nonlinear dynamical systems 
using the phase-space geometric structures. More specifically, we define control- 
lability, stability, robustness, and optimality of control systems geometrically so 
that these properties can be operationally verified. 

— The controllability criterion tests if a particular target state is reachable 
from a given initial state or an operating region of a system under a control 
signal. The set of reachable states form a subspace of phase space that can 
be computed geometrically. 

— The stability criterion tests if a system remains within the neighborhood of 
a target state for a reasonable amount of time. Stability is characterized by 
the stability region — the set of initial states that evolve to the same limit set. 
The stability region is a subspace of phase space that can also be computed 
geometrically. 

— The robustness criterion checks if a system attains the same properties when 
parametric or structural uncertainties are introduced. Geometrically, certain 
types of uncertainties like noise or measurement errors can be modeled as 
regions around states or a sequence of states. 
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components 






Parameterized phase-space vector Reids 



Parameterized phase-space vector fields 



Fig. 1. Spatial aggregation of phase-space control trajectories. The phase-space 
trajectories, parameterized for a particular control action, arc first aggregated 
into a neighborhood graph and classified into behavioral groups according to 
various definitions of behavioral equivalence. Given admissible control actions, 
the classified neighborhood graphs are aggregated to form a composite neighbor- 
hood graph which can be then be classified and searched out to identify optimal 
control policies. 
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— The optimality criterion tests if the system achieves the goal with an op- 
timal amount of resources such as time and energy. The consideration on 
the resource consumption parameterizes phase-space trajectories. Control 
reference trajectories arc ranked according to prcspccificd optimality metric. 

The phase-space geometric definition of controllability and robustness be- 
comes the basis for developing the experimental evaluation methods for the 
maglev control system. 

3 The Maglev Control Experiment 



Fig. 2. (a) German Transrapid magnetically levitated transportation system, 
(b) A maglev vehicle and guideway system with active primary and secondary 
suspensions, subject to aerodynamic disturbances and guideway variations. Such 
a system, often highly nonlinear, requires a high-performance controller to ensure 
ride smoothness. 

Magnetically levitated vehicles such as the German Transrapid train (Fig- 
ure 2(a)) require active suspension technology that maintains a constant air gap 
and dampens disturbances caused by road irregularities and wind loads [6]. Fig- 
ure 2(b) shows a maglev vehicle and guideway system with active primary and 
secondary suspensions [13]. In such a system, the control objective for the active 
suspension system is to ensure a good ride quality for passengers on realistic 
guideway systems. 

In the earlier work, we had developed a phase-space control algorithm that 
searches through equivalence classes of trajectories to synthesize a global switch- 
ing control law [21,23]. Simulation result shows that the synthesized controller 
compares favorably to classical linear feedback design and does not require linear 
approximations to the model dynamics. The current maglev control project aims 
at developing practical phase-space control algorithms and implementations for 
a prototype of maglev system whose dynamics resembles that of the German 
Ti'ansrapid. The implemented control system monitors the state of the maglev 
system, computes the required control action, and actuates an electromagnet to 
counter disturbances. 




(a) 



(b) 
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3.1 A maglev control system prototype 



We have constructed a prototype for the maglev control experiment in the lab, 
shown in Figure 3. The system uses an electromagnet to suspend a steel ball 
in the air and is representative of magnetically levitated transportation systems 
(EMS system) such as the German Transrapid system. At the equilibrium point, 
the attracting magnetic force balances the gravitational force acting on the ball. 
However, this attractive system is inherently unstable^. An active controller is 
required to maintain a constant air gap between the ball and the magnet in the 
presence of disturbances. 

The block diagram of the experimental control system is shown in Figure 4. 
The data for ball displacement, velocity, and solenoid current is collected through 
photo sensors and current sensor. A 12-bit analog-to-digital converter samples 
the data at a rate of about 5000Hz. A digital computer (Pentium 75MHz) imple- 
ments the control algorithm and a low-pass filter, and provides a run-time user 
interface. The digital controller employs either a global or a local control algo- 
rithm, depending on the current state in phase space. The appropriate control 
signal is delivered to the digital-to-analog converter, amplified, and then applied 
to the electro-magnet that suspends the ball. While the control is in progress, 
the user can interrupt the system through the user interface for tasks such as 
introducing disturbance. 




Fig. 3. The maglev control system prototype: (a) Photo of the physical experi- 
ment. (b) Basic components. 



^ In contrast, an EDS maglev system uses repulsive magnetic force to suspend vehicles 
and is inherently stable. However, an EDS system requires superconducting circuits 
in order to reduce energy loss. 
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Fig. 4. Block diagram for the maglev control system. 



3.2 The maglev model 



The nonlinear model for the maglev system is described by 



= V 



dx 
dt 



LqXqI^ 

2mx^ 



( 1 ) 



where the state variables x and w represent the vertical gap between steel ball and 
magnet and the vertical velocity of the ball, respectively. The control parameter 
is the coil current I. The other parameters are the ball mass m = 0.008432/fgr, 
the solenoid-ball system inductance Lq — 0.00802i/ at the equilibrium, the de- 
sired equilibrium vertical gap xq — 0.0066m, and the gravitational acceleration 
g = 9.81m/s^. The nonlinearity of the system comes from the inverse square 
magnetic force law. 

At the desired equilibrium gap xq, there is a unique coil current Iq for which 
the magnetic force exactly counterbalances the force due to gravity and the ball 
has no vertical velocity and acceleration. However, the equilibrium is a saddle 
node which is not stable. The control objective, therefore, is to stabilize the ball 
and maintain a constant air gap despite any disturbances. The available control 
input is the coil current I in the model (1) provided by a voltage-controlled 
current source, varying from 0.03A to 0.83A. The power supply delivers /q = 
0.38A at the equilibrium. 



4 Phase-Space Control Trajectory Design: an algorithm 

The phase-space based control algorithm determines a control law for a discrete 
sampled system. The algorithm assumes that the control objective is to move 
the system to a prespecified goal state (or set of states) and that the behavior 
of the system is governed by the following map: 

^'n+l T(x^, cj 

where is the state vector of the system at a given time, c is the control in- 
put vector, and is the state of the system after one sampling period (it is 
assumed that the sampling period is fixed) . If the behavior of the system is de- 
scribed in continuous terms, the discrete equation can be obtained by integrating 
the continuous equation over one sampling period. 
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The algorithm takes as input the model for the system, the goal state, the set 
of allowable control values, and information about how to partition the phase 
space. It outputs a table of control actions, with each cell having its own entry 
in the table. The runtime control program determines which cell contains the 
current system state and indexes into the control table to retrieve the appropriate 
control output. 

The algorithm consists of three main steps. 

1. Partition the phase space into cellular regions^. 

2. Compute the graph of cell adjacencies. A cell x is said to be adjacent to a 

cell y if, for some allowable control value c, there exists a natural number n 
such that {center {x) ^ c) G x,Vfc < n and {center {x),c) G y. 

3. For each cell c, find a path to the goal cell (i.e. the cell containing the goal 
state). If no such path exists, c is marked as outside the controllable region 
of the system. Otherwise, choose one such path, and enter the control output 
corresponding to this path into the control table. 

Step three employs path selection algorithms to choose a path to the goal 
cell when multiple such paths exist. Typically, path selection will be based on 
system performance criteria. For example, if a fast system response is desired, 
short paths can be chosen over longer paths. 



4.1 Phase-Space Control Performance Evaluation 

Most nonlinear systems are not amenable to analytic characterization of con- 
trol performance such as controllability and robustness. The phase-space con- 
trol method is particularly well suited for synthesizing control for this class of 
systems. The phase-space interpretation of control properties provides a basis 
for computational implementation of practical control performance evaluation 
methods for nonlinear systems. These evaluation methods can be used to ex- 
perimentally validate if an implementation satisfies the design speeifieation and 
eompare different control strategies. 

— Controllable region and robustness: 

The controllable region is defined as the collection of cells from which the goal 
states are reachable. Operationally, the region is characterized by aggregating 
cell transitions using measured trajectories. The controllable region can be 
labeled with sizes and other more refined performance characterization. 
Robustness measures the ability of the system to withstand disturbances. 
Experimentally, we characterize for each state of the system the magnitude 
and duration of destabilizing disturbance. In the experiment, the disturbance 
is introduced as part of the control input to the system. 

A system with a larger controllable region is generally more stable and robust 
to disturbances such as a sudden displacement. 

The phase-space partition docs not have to be uniform. 
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— Settling time, rise time, and overshoot: 

Settling time is defined as the time for the system to settle in the vicinity 
of the steady state. That is, it measures the time required for the transient 
state to decay. On the other hand, rise time is the time for the system output 
to reach the vicinity of the desired point for the first time, and measures 
the response time of the system. Overshoot is the maximum amount the 
system output overshoots the final value. The amount of overshoot is a good 
indication of the smoothness of control and is normally measured as the 
percent relative to the final value. A control design typically requires trade- 
off between response time and overshooting. 

Experimentally, we define a set of iso-curves for settling time with respect 
to a steady state in phase space, shown in Figure 5. Rise time is likewise 
characterized. Overshoot is indirectly characterized by the curvature along 
trajectories in the neighborhood of an equilibrium. 




Fig. 5. Iso-curves for settling time with respect to a stable equilibrium state, 
shown in phase space. 



5 Experimental Results and Analysis 

We have implemented and evaluated the phase-space control algorithm on the 
maglev system. In the experiment, the phase space of the maglev system is par- 
titioned into a 50 by 50 uniform cellular space. The control space is also sampled 
to form a discrete control set. The path search algorithm uses an iterative order 
cn? method (where n is the number of cells in the phase space discretization and 
c is the number of divisions in the control space) that does not require compu- 
tation of all possible paths. Paths are selected on the basis of two criteria. The 
first criteria gives preference to shorter paths (with respect to the number of cells 
contained in the path) over longer paths. The second criteria gives preference 
to paths that are more robust with respect to disturbances or modeling errors. 
More specifically, it prefers paths that are further away from the “behavioral 
boundaries” of the system. For our purposes, a behavioral boundary in phase 
space is an area of phase space where the field direction in neighboring cells dif- 
fers substantially (Figure 6). Paths that use cells close to a behavioral boundary 
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are more susceptible to disturbances or modeling error, since a small error can 
result in a large change in the expected trajectory near these boundaries. 



Cioal 




Fig. 6. Two adjacent phase-space cells at a behavioral boundary. Each arrow, 
for a given control value, indicates the direction of the field at the cell center. 
The two shaded cells lie on a behavioral boundary since their field directions 
differ substantially. 



Figure 7 shows the control graph generated for the magnetic levitation sys- 
tem. The control table is computed off-line. Real-time control is accomplished 
by fast sensing, state estimation, and control action lookup. The phase-space 
control incorporates a local control that takes over when the system enters the 
neighborhood of the equilibrium. 

The experimental results on the physical maglev system show that the glob- 
al, phase-space control algorithm is able to suspend the steel ball reliably under 
normal operating conditions for at least 15 minutes (after 15 minutes the ap- 
paratus must be turned oft to avoid overheating of the solenoid). We have also 
introduced external disturbances into the system (in the form of a momentary 
current drop that results in a sudden change in air gap and velocity) and exper- 
imentally measured the robustness of the system. 

The phase space plot of the controlled trajectory is shown in Figure 8. Fig- 
ure 9 shows how the air gap, velocity, and control vary over time. The initial 
bump in the air gap plot is due to the disturbance artificially introduced. The 
controller is activated to stabilize the system as soon as the disturbance is de- 
tected. 

Using the phase-space performance evaluation method, the controllable re- 
gion for the global controller is depicted in Figure 10. 

6 Related work 

The field of nonlinear control has developed a rich collection of design methods 
that apply to specific classes of problems [16]. For instance, methods of lineariza- 
tion and describing functions generalize linear techniques to nonlinear systems. 
The technique of gain scheduling, for example, approximates a nonlinear con- 
trol system with a piccewise-linear one and designs a linear controller for each 
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Control Graph Plot 




velocity (m/s) 



Fig. 7. Synthesized control paths (to the goal) are plotted against phase space, 
with lines indicating paths traveled for each cell. The phase space is divided 
into a uniform 20 by 20 grid for the purpose of visualization. Continuous lines 
indicate the chosen path from a cell to the goal state. The height of a point along 
a path corresponds to the magnitude of the control input at that location. 




5 5.5 6 6.5 7 7.5 8 8.5 9 

Position (m) x10"^ 



Fig. 8. A sample maglev control trajectory shown in phase space (position D 
velocity). The equilibrium point (0.0066, 0) is inside the dense region covered by 
the orbit. Notice that the global controller is able to bring the trajectory back 
to equilibrium after an initial disturbance. 
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Fig. 9. Experimental data from the maglev system: (a) Disturbance introduced 
at the beginning, (b) Air gap x vs. time. Notice that the system stabilizes after 
the initial disturbance, (c) Velocity v vs. time, (d) Control I vs. time. The global, 
phase-space controller is activated whenever the position or velocity of the ball 
deviates from the desired value significantly. When the system is within the 
capture region of the local control, the controller switches to the local control. 
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Fig. 10. Region of controllability for the phase-space based global control law, 
comprising cells marked by D. 
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linear piece. To apply the gain scheduling technique, one has to decompose the 
phase space into locally linear regions and design a linear controller for each 
region. Phase-space control generalizes the decomposition process to permit a 
systematic computational exploration. 

Hsu developed a cell-based method called cell-to-cell mapping for approxi- 
mating state spaces [10]. A continuous state space is discretized into regular cells 
forming a cell space. The associated map of a system becomes a cell-to-cell map 
which maps one cell to another cell. The cell-to-cell mapping method approx- 
imates the stability region of an attracting cell with a collection of cells that 
eventually map to that cell. Our phase-space control algorithm goes a major 
step beyond the cell-based analysis. We had previously developed a phase-space 
control framework for exploiting phase-space global knowledge to obtain high- 
quality control design [2]. This work focuses on developing practical methods and 
implementations for the phase-space control technology. We use the cell decom- 
position as a first order approximation to the phase-space geometry to develop 
practical control synthesis and evaluation methods. The cellular neighborhood 
structure serves as a place holder for phase-space control performance data such 
as resource consumption, controllability, and robustness that can be actively ex- 
ploited in synthesizing high-quality control actions. Additionally, the structure 
permits programmed trade-offs between computation and control quality. 

Research in hybrid systems, a class of dynamical systems that possess both 
continuous dynamics and discrete transitions, has produced a body of theories 
and algorithms for control analysis and synthesis [1,5,3,4,8,14,17]. Phase-space 
control systems form a special class of hybrid systems. The formal results ob- 
tained in hybrid systems research provide theoretical characterizations for the 
phase-space control systems. We have focused on developing practical algorithm- 
s for synthesizing control actions for nonlinear systems, using phase space as a 
geometric model. We have also characterized the computational complexity for 
a restricted class of hybrid systems [12]. 

Recently, neural net technology has been successfully applied to the control 
of highly nonlinear systems. In one case, a neural net is used to model the inverse 
dynamics of a high-performance fighter jet in order to produce high-quality con- 
trol design [18]. However, one major drawback of the neural-net based controller 
is its lack of stability and performance guarantee required by the FAA certifica- 
tion process. The tools described in this paper can computationally characterize 
operational properties of these nonlinear controllers that are not amenable to 
traditional analytical analysis. Hence, our approach is complementary to exist- 
ing nonlinear control technology. 

7 Conclusion 

We have described the Phase-Space Nonlinear Control Toolbox for synthesiz- 
ing and evaluating control laws in phase space. We have demonstrated, using 
the maglev experiment, that the phase-space control technology is feasible. The 
current capability of the Toolbox is limited in a number of ways. We plan to in- 
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corporate and exploit additional global phase-space features, new optimization 
metrics, statistics-based evaluation methods, and learning into the phase-space 
framework. The phase-space control is applicable to a broad class of physical sys- 
tems operating in large nonlinear regimes, for which conventional linear, analytic 
methods are ill-suited. 

The rapid advances in information processing, micro-fabrication, and MEMS 
are fueling a new generation of distributed autonomous systems [19]. Our a- 
bilities to sense and act in the complex physical environments are increasingly 
augmented by massive networks of tiny, invisible sensors, actuators, and comput- 
ers embedded in everything from appliances to materials to building structures. 
These distributed computational agents are immersed in physical media and 
governed by the fundamental laws of computation and physics. Computational 
methods such as phase-space control may enable these agents to maximumly ex- 
ploit the environment at the juncture of digital universe and continuous physical 
world. 
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